2023-10-30T06:02:59.125 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-10-30T06:02:59.126 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_sjust@teuthology --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind} --name sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi --archive /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005 -- /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/orig.config.yaml 2023-10-30T06:02:59.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T06:02:59.247 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005 branch: main description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind} email: null first_in_suite: false job_id: '7441005' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - \(SMALLER_PGP_NUM\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW OPS\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: d0db9f2f95b9d6344950185071cace2a50d296eb valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: d0db9f2f95b9d6344950185071cace2a50d296eb thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: d0db9f2f95b9d6344950185071cace2a50d296eb owner: scheduled_sjust@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5205 sha1: d0db9f2f95b9d6344950185071cace2a50d296eb sleep_before_teardown: 0 subset: 220/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d0db9f2f95b9d6344950185071cace2a50d296eb targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOy9njApuL9486j3mpSSWwFrt0UsaVb0q/6Gn4YBG+PqZmwquojCA12rs3VU4DVsLcQDMY61NwRuS+U4/5jWhEU= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6npLNQ3Gfa1m3akue2LinVUVmcPcQi3rhjOXz5RrAMVi0QhDKvV/Xf1DeWYnn3MpsGFcqGrbCo/2Pe0QXkCw= tasks: - install: null - ceph: null - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - mon_recovery: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-10-30_05:47:25 tube: smithi user: sjust verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-30T06:02:59.248 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa; will attempt to use it 2023-10-30T06:02:59.249 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks 2023-10-30T06:02:59.249 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-30T06:02:59.251 INFO:teuthology.task.internal:Checking packages... 2023-10-30T06:02:59.277 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0db9f2f95b9d6344950185071cace2a50d296eb' 2023-10-30T06:02:59.278 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-30T06:02:59.278 INFO:teuthology.packaging:ref: None 2023-10-30T06:02:59.279 INFO:teuthology.packaging:tag: None 2023-10-30T06:02:59.279 INFO:teuthology.packaging:branch: main 2023-10-30T06:02:59.279 INFO:teuthology.packaging:sha1: d0db9f2f95b9d6344950185071cace2a50d296eb 2023-10-30T06:02:59.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-10-30T06:02:59.589 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6863.g636384c4 2023-10-30T06:02:59.590 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-30T06:02:59.597 INFO:teuthology.task.internal:no buildpackages task found 2023-10-30T06:02:59.597 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-30T06:02:59.617 INFO:teuthology.task.internal:Saving configuration 2023-10-30T06:02:59.637 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-30T06:02:59.643 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T06:02:59.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 05:57:54.528055', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOy9njApuL9486j3mpSSWwFrt0UsaVb0q/6Gn4YBG+PqZmwquojCA12rs3VU4DVsLcQDMY61NwRuS+U4/5jWhEU='} 2023-10-30T06:02:59.693 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 05:57:54.529735', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6npLNQ3Gfa1m3akue2LinVUVmcPcQi3rhjOXz5RrAMVi0QhDKvV/Xf1DeWYnn3MpsGFcqGrbCo/2Pe0QXkCw='} 2023-10-30T06:02:59.693 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-30T06:02:59.699 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-10-30T06:02:59.699 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-10-30T06:02:59.699 INFO:teuthology.run_tasks:Running task console_log... 2023-10-30T06:02:59.779 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbd9b468a60>, signals=[15]) 2023-10-30T06:02:59.781 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-30T06:02:59.787 INFO:teuthology.task.internal:Opening connections... 2023-10-30T06:02:59.787 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:02:59.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:02:59.877 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:02:59.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:02:59.962 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-30T06:02:59.969 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2023-10-30T06:02:59.987 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2023-10-30T06:02:59.988 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-10-30T06:03:00.044 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2023-10-30T06:03:00.044 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2023-10-30T06:03:00.044 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2023-10-30T06:03:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2023-10-30T06:03:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2023-10-30T06:03:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2023-10-30T06:03:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-30T06:03:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-10-30T06:03:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2023-10-30T06:03:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-30T06:03:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2023-10-30T06:03:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-30T06:03:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-30T06:03:00.047 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-30T06:03:00.047 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-10-30T06:03:00.109 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-10-30T06:03:00.127 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-10-30T06:03:00.127 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-10-30T06:03:00.183 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2023-10-30T06:03:00.184 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2023-10-30T06:03:00.184 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2023-10-30T06:03:00.184 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2023-10-30T06:03:00.185 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2023-10-30T06:03:00.185 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2023-10-30T06:03:00.185 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-30T06:03:00.185 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-10-30T06:03:00.185 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2023-10-30T06:03:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-30T06:03:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2023-10-30T06:03:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-30T06:03:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-30T06:03:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-30T06:03:00.187 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-10-30T06:03:00.207 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-30T06:03:00.215 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-30T06:03:00.221 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-30T06:03:00.222 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2023-10-30T06:03:00.225 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2023-10-30T06:03:00.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-30T06:03:00.248 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-30T06:03:00.248 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2023-10-30T06:03:00.281 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2023-10-30T06:03:00.384 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-30T06:03:00.443 INFO:teuthology.run_tasks:Running task kernel... 2023-10-30T06:03:00.459 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-30T06:03:00.460 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-30T06:03:00.460 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T06:03:00.461 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-30T06:03:00.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:03:00.478 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2023-10-30T06:03:00.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:03:00.481 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-10-30T06:03:00.536 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-361.el9.x86_64 2023-10-30T06:03:00.536 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-361.el9.x86_64 2023-10-30T06:03:00.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2023-10-30T06:03:00.538 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-361.el9.x86_64 2023-10-30T06:03:00.538 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-361.el9.x86_64 2023-10-30T06:03:00.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2023-10-30T06:03:02.792 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-30T06:03:02.800 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-30T06:03:04.184 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2023-10-30T06:03:04.484 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2023-10-30T06:03:05.008 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 7.9 MB 00:00 2023-10-30T06:03:06.091 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 7.9 MB 00:01 2023-10-30T06:03:07.842 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 25 MB/s | 18 MB 00:00 2023-10-30T06:03:11.943 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 5.7 MB 00:00 2023-10-30T06:03:13.385 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2023-10-30T06:03:14.420 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 21 MB/s | 19 MB 00:00 2023-10-30T06:03:16.765 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 18 MB 00:10 2023-10-30T06:03:18.945 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2023-10-30T06:03:21.035 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-30T06:03:21.036 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-30T06:03:21.124 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:03:21.135 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:03:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2023-10-30T06:03:21.135 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-378.el9 baseos 5.3 M 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-378.el9 baseos 20 M 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-378.el9 baseos 38 M 2023-10-30T06:03:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-378.el9 baseos 32 M 2023-10-30T06:03:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:03:21.137 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:03:21.137 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:03:21.137 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2023-10-30T06:03:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:03:21.138 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 95 M 2023-10-30T06:03:21.138 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 120 M 2023-10-30T06:03:21.139 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:03:21.190 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 5.7 MB 00:01 2023-10-30T06:03:21.830 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-378.el9.x86_64.rpm 13 MB/s | 5.3 MB 00:00 2023-10-30T06:03:22.255 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-378.el9.x86_64.rpm 24 MB/s | 20 MB 00:00 2023-10-30T06:03:22.499 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 15 kB 00:00 2023-10-30T06:03:23.014 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-5.14.0-378.el9.x86_64.rpm 23 MB/s | 38 MB 00:01 2023-10-30T06:03:23.392 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-core-5.14.0-378.el9.x86_6 21 MB/s | 32 MB 00:01 2023-10-30T06:03:23.395 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:03:23.395 INFO:teuthology.orchestra.run.smithi112.stdout:Total 42 MB/s | 95 MB 00:02 2023-10-30T06:03:24.239 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:03:24.593 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:03:24.593 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:03:25.916 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:03:25.917 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:03:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:03:29.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-378.el9.x86_64 1/4 2023-10-30T06:03:29.331 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:03:29.716 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 19 MB 00:07 2023-10-30T06:03:32.058 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:03:32.275 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:03:34.324 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2023-10-30T06:03:36.372 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-10-30T06:03:36.372 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-10-30T06:03:36.471 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:03:36.480 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:03:36.480 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2023-10-30T06:03:36.481 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:03:36.481 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:03:36.481 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-378.el9 baseos 5.3 M 2023-10-30T06:03:36.481 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:03:36.481 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core x86_64 5.14.0-378.el9 baseos 20 M 2023-10-30T06:03:36.482 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules x86_64 5.14.0-378.el9 baseos 38 M 2023-10-30T06:03:36.482 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core x86_64 5.14.0-378.el9 baseos 32 M 2023-10-30T06:03:36.482 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:03:36.483 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:03:36.483 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:03:36.483 INFO:teuthology.orchestra.run.smithi079.stdout:Install 4 Packages 2023-10-30T06:03:36.483 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:03:36.484 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 95 M 2023-10-30T06:03:36.484 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 120 M 2023-10-30T06:03:36.485 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:03:36.865 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:03:36.973 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:03:37.000 INFO:teuthology.orchestra.run.smithi079.stdout:(1/4): kernel-5.14.0-378.el9.x86_64.rpm 12 MB/s | 5.3 MB 00:00 2023-10-30T06:03:37.617 INFO:teuthology.orchestra.run.smithi079.stdout:(2/4): kernel-core-5.14.0-378.el9.x86_64.rpm 19 MB/s | 20 MB 00:01 2023-10-30T06:03:38.059 INFO:teuthology.orchestra.run.smithi079.stdout:(3/4): kernel-modules-5.14.0-378.el9.x86_64.rpm 25 MB/s | 38 MB 00:01 2023-10-30T06:03:38.726 INFO:teuthology.orchestra.run.smithi079.stdout:(4/4): kernel-modules-core-5.14.0-378.el9.x86_6 19 MB/s | 32 MB 00:01 2023-10-30T06:03:38.726 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:03:38.726 INFO:teuthology.orchestra.run.smithi079.stdout:Total 42 MB/s | 95 MB 00:02 2023-10-30T06:03:39.567 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:03:39.947 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:03:39.948 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:03:41.252 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:03:41.253 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:03:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:03:43.888 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:03:44.515 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-core-5.14.0-378.el9.x86_64 1/4 2023-10-30T06:03:44.659 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:03:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:03:47.610 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:03:52.238 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:03:52.358 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : kernel-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:03:57.008 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-378.el9.x86_64 1/4 2023-10-30T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:04:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:04:11.480 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-378.el9.x86_64 2023-10-30T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-378.el9.x86_64 2023-10-30T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-378.el9.x86_64 2023-10-30T06:04:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:04:11.482 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:04:12.015 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2023-10-30T06:04:12.156 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:12.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-10-30T06:04:12.156 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-378.el9.x86_64 2023-10-30T06:04:12.157 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-30T06:04:12.157 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-30T06:04:12.157 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-30T06:04:12.158 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2023-10-30T06:04:12.158 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2023-10-30T06:04:12.275 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:12.275 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-10-30T06:04:12.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2023-10-30T06:04:12.320 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2023-10-30T06:04:12.320 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2023-10-30T06:04:12.320 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2023-10-30T06:04:12.321 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2023-10-30T06:04:12.321 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2023-10-30T06:04:12.321 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-30T06:04:12.321 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2023-10-30T06:04:12.321 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2023-10-30T06:04:12.322 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2023-10-30T06:04:12.322 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-30T06:04:12.322 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-30T06:04:12.322 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-30T06:04:12.322 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-30T06:04:12.323 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2023-10-30T06:04:12.323 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2023-10-30T06:04:12.323 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-30T06:04:12.323 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2023-10-30T06:04:12.323 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2023-10-30T06:04:12.324 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:04:12.325 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2023-10-30T06:04:12.326 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-30T06:04:12.326 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-378.el9.x86_64 2023-10-30T06:04:12.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-30T06:04:12.884 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2023-10-30T06:04:14.246 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-30T06:04:14.278 INFO:teuthology.orchestra.run.smithi112.stderr:done 2023-10-30T06:04:14.280 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-10-30T06:04:14.297 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.wkpHLnZpy0 2023-10-30T06:04:14.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wkpHLnZpy0 2023-10-30T06:04:14.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.wkpHLnZpy0 2023-10-30T06:04:14.511 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.wkpHLnZpy0 is 6KB 2023-10-30T06:04:14.561 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.wkpHLnZpy0 2023-10-30T06:04:14.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2023-10-30T06:04:14.639 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-30T06:04:14.640 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-30T06:04:14.640 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-30T06:04:14.640 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64.conf 2023-10-30T06:04:14.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64 2023-10-30T06:04:14.784 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2023-10-30T06:04:26.317 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:26.358 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-modules-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:26.747 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-378.el9.x86_64 1/4 2023-10-30T06:04:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-core-5.14.0-378.el9.x86_64 2/4 2023-10-30T06:04:26.748 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-5.14.0-378.el9.x86_64 3/4 2023-10-30T06:04:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-modules-core-5.14.0-378.el9.x86_64 4/4 2023-10-30T06:04:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:04:27.178 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:04:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-core-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-modules-core-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:04:27.179 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:04:27.685 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2023-10-30T06:04:27.836 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.836 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-10-30T06:04:27.836 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-378.el9.x86_64 2023-10-30T06:04:27.837 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-30T06:04:27.837 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-30T06:04:27.837 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-30T06:04:27.837 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi079.front.sepia.ceph.com, path=None, version=distro) 2023-10-30T06:04:27.838 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2023-10-30T06:04:27.954 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-378.el9.x86_64 2023-10-30T06:04:27.954 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-378.el9.x86_64 2023-10-30T06:04:27.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo rpm -qi grub2-tools 2023-10-30T06:04:27.988 INFO:teuthology.orchestra.run.smithi079.stdout:Name : grub2-tools 2023-10-30T06:04:27.989 INFO:teuthology.orchestra.run.smithi079.stdout:Epoch : 1 2023-10-30T06:04:27.989 INFO:teuthology.orchestra.run.smithi079.stdout:Version : 2.06 2023-10-30T06:04:27.989 INFO:teuthology.orchestra.run.smithi079.stdout:Release : 68.el9 2023-10-30T06:04:27.989 INFO:teuthology.orchestra.run.smithi079.stdout:Architecture: x86_64 2023-10-30T06:04:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-30T06:04:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:Group : Unspecified 2023-10-30T06:04:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:Size : 8264254 2023-10-30T06:04:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:License : GPLv3+ 2023-10-30T06:04:27.990 INFO:teuthology.orchestra.run.smithi079.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-30T06:04:27.991 INFO:teuthology.orchestra.run.smithi079.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-30T06:04:27.991 INFO:teuthology.orchestra.run.smithi079.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-30T06:04:27.991 INFO:teuthology.orchestra.run.smithi079.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-30T06:04:27.991 INFO:teuthology.orchestra.run.smithi079.stdout:Packager : builder@centos.org 2023-10-30T06:04:27.991 INFO:teuthology.orchestra.run.smithi079.stdout:Vendor : CentOS 2023-10-30T06:04:27.992 INFO:teuthology.orchestra.run.smithi079.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-30T06:04:27.992 INFO:teuthology.orchestra.run.smithi079.stdout:Summary : Support tools for GRUB. 2023-10-30T06:04:27.992 INFO:teuthology.orchestra.run.smithi079.stdout:Description : 2023-10-30T06:04:27.992 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:04:27.992 INFO:teuthology.orchestra.run.smithi079.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-30T06:04:27.993 INFO:teuthology.orchestra.run.smithi079.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-30T06:04:27.993 INFO:teuthology.orchestra.run.smithi079.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-30T06:04:27.993 INFO:teuthology.orchestra.run.smithi079.stdout:hardware devices. 2023-10-30T06:04:27.993 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:04:27.993 INFO:teuthology.orchestra.run.smithi079.stdout:This subpackage provides tools for support of all platforms. 2023-10-30T06:04:27.994 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-30T06:04:27.995 INFO:teuthology.task.kernel:Updating grub on smithi079 to boot 5.14.0-378.el9.x86_64 2023-10-30T06:04:27.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-30T06:04:28.718 INFO:teuthology.orchestra.run.smithi079.stderr:Generating grub configuration file ... 2023-10-30T06:04:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-30T06:04:30.178 INFO:teuthology.orchestra.run.smithi079.stderr:done 2023-10-30T06:04:30.181 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2023-10-30T06:04:30.237 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.FmMM9Kw2zA 2023-10-30T06:04:30.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FmMM9Kw2zA 2023-10-30T06:04:30.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.FmMM9Kw2zA 2023-10-30T06:04:30.445 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.FmMM9Kw2zA is 6KB 2023-10-30T06:04:30.493 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.FmMM9Kw2zA 2023-10-30T06:04:30.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo /bin/ls /boot/loader/entries || true 2023-10-30T06:04:30.572 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-30T06:04:30.572 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-30T06:04:30.573 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-30T06:04:30.573 INFO:teuthology.orchestra.run.smithi079.stdout:15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64.conf 2023-10-30T06:04:30.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-378.el9.x86_64 2023-10-30T06:04:30.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo shutdown -r now 2023-10-30T06:04:44.800 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-30T06:04:44.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:04:44.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:00.722 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-30T06:05:00.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:05:00.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:03.226 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-10-30T06:05:12.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:05:12.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:15.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-10-30T06:05:19.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-30T06:05:27.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:05:27.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:28.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:05:28.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:31.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-30T06:05:43.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:05:43.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:05:59.108 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-10-30T06:05:59.474 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:05:59.475 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-378.el9.x86_64"... 2023-10-30T06:05:59.475 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-10-30T06:05:59.489 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-378.el9.x86_64 2023-10-30T06:05:59.489 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-378.el9.x86_64 vs 5.14.0-378.el9.x86_64 2023-10-30T06:05:59.489 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-30T06:05:59.490 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-30T06:06:00.491 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-30T06:06:00.491 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-30T06:06:00.548 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2023-10-30T06:06:00.586 DEBUG:teuthology.parallel:result is None 2023-10-30T06:06:15.147 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-10-30T06:06:15.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:06:15.607 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-378.el9.x86_64"... 2023-10-30T06:06:15.607 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2023-10-30T06:06:15.631 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-378.el9.x86_64 2023-10-30T06:06:15.631 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-378.el9.x86_64 vs 5.14.0-378.el9.x86_64 2023-10-30T06:06:15.631 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-30T06:06:15.632 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-30T06:06:16.633 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-30T06:06:16.633 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-30T06:06:16.776 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2023-10-30T06:06:17.009 DEBUG:teuthology.parallel:result is None 2023-10-30T06:06:17.009 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-30T06:06:17.016 INFO:teuthology.task.internal:Creating test directory... 2023-10-30T06:06:17.017 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T06:06:17.022 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-30T06:06:17.041 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-30T06:06:17.050 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-30T06:06:17.059 INFO:teuthology.task.internal:Creating archive directory... 2023-10-30T06:06:17.060 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T06:06:17.077 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-30T06:06:17.153 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-30T06:06:17.160 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-30T06:06:17.160 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T06:06:17.162 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-30T06:06:17.201 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T06:06:17.213 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T06:06:17.214 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T06:06:17.226 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-30T06:06:17.227 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-30T06:06:17.236 INFO:teuthology.task.internal:Configuring sudo... 2023-10-30T06:06:17.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T06:06:17.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-30T06:06:17.285 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-30T06:06:17.295 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-30T06:06:17.296 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T06:06:17.305 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-30T06:06:17.340 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T06:06:17.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T06:06:17.457 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T06:06:17.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T06:06:17.579 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:06:17.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T06:06:17.645 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T06:06:17.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-30T06:06:17.740 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T06:06:17.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-30T06:06:17.866 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:06:17.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-30T06:06:17.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2023-10-30T06:06:17.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2023-10-30T06:06:17.977 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-30T06:06:18.015 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-30T06:06:18.526 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-30T06:06:18.534 INFO:teuthology.task.internal:Starting timer... 2023-10-30T06:06:18.534 INFO:teuthology.run_tasks:Running task pcp... 2023-10-30T06:06:18.546 INFO:teuthology.run_tasks:Running task selinux... 2023-10-30T06:06:18.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2023-10-30T06:06:18.612 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2023-10-30T06:06:18.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2023-10-30T06:06:18.675 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2023-10-30T06:06:18.677 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-30T06:06:18.677 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2023-10-30T06:06:18.698 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2023-10-30T06:06:18.698 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2023-10-30T06:06:18.729 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2023-10-30T06:06:18.729 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2023-10-30T06:06:18.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-30T06:06:18.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:06:18.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-30T06:06:18.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:06:18.797 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-30T06:06:18.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2023-10-30T06:06:18.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2023-10-30T06:06:18.881 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-30T06:06:18.892 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-30T06:06:18.892 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-30T06:06:18.911 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-30T06:06:18.913 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-30T06:11:41.379 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2023-10-30T06:11:41.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:11:41.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:11:41.471 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-10-30T06:11:41.497 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-30T06:11:41.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:11:41.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-30T06:11:41.582 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-10-30T06:11:41.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-10-30T06:11:41.649 INFO:teuthology.run_tasks:Running task clock... 2023-10-30T06:11:41.661 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-30T06:11:41.662 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T06:11:41.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T06:11:41.665 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-30T06:11:41.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T06:11:41.694 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-30T06:11:41.711 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-30T06:11:41.720 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-30T06:11:41.737 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-30T06:11:41.741 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2023-10-30T06:11:41.754 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2023-10-30T06:11:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2023-10-30T06:11:41.770 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-30T06:11:41.778 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2023-10-30T06:11:41.786 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-30T06:11:41.795 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-30T06:11:41.811 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-30T06:11:41.829 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2023-10-30T06:11:41.833 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-30T06:11:41.833 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2023-10-30T06:11:41.833 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.833 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.834 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.834 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.853 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2023-10-30T06:11:41.856 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-30T06:11:41.856 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-10-30T06:11:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.857 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:11:41.858 INFO:teuthology.run_tasks:Running task install... 2023-10-30T06:11:41.867 DEBUG:teuthology.task.install:project ceph 2023-10-30T06:11:41.867 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'}} 2023-10-30T06:11:41.868 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'} 2023-10-30T06:11:41.868 INFO:teuthology.task.install:Using flavor: default 2023-10-30T06:11:41.874 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2023-10-30T06:11:41.874 INFO:teuthology.task.install:extra packages: [] 2023-10-30T06:11:41.875 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'tag': None, 'wait_for_package': False} 2023-10-30T06:11:41.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0db9f2f95b9d6344950185071cace2a50d296eb 2023-10-30T06:11:41.877 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'tag': None, 'wait_for_package': False} 2023-10-30T06:11:41.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0db9f2f95b9d6344950185071cace2a50d296eb 2023-10-30T06:11:42.098 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/ 2023-10-30T06:11:42.099 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6858.gd0db9f2f 2023-10-30T06:11:42.103 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/ 2023-10-30T06:11:42.104 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6858.gd0db9f2f 2023-10-30T06:11:42.208 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-30T06:11:42.209 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:11:42.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-30T06:11:42.223 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/d0db9f2f95b9d6344950185071cace2a50d296eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-30T06:11:42.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:11:42.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-30T06:11:42.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-10-30T06:11:42.237 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0db9f2f95b9d6344950185071cace2a50d296eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-30T06:11:42.252 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-10-30T06:11:42.252 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0db9f2f95b9d6344950185071cace2a50d296eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-30T06:11:42.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-30T06:11:42.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-30T06:11:42.385 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-30T06:11:42.401 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-30T06:11:42.451 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2023-10-30T06:11:42.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2023-10-30T06:11:42.467 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2023-10-30T06:11:42.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2023-10-30T06:11:42.699 INFO:teuthology.orchestra.run.smithi112.stdout:72 files removed 2023-10-30T06:11:42.702 INFO:teuthology.orchestra.run.smithi079.stdout:72 files removed 2023-10-30T06:11:42.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw 2023-10-30T06:11:42.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw 2023-10-30T06:11:43.372 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 238 kB/s | 78 kB 00:00 2023-10-30T06:11:43.435 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 201 kB/s | 78 kB 00:00 2023-10-30T06:11:43.797 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-10-30T06:11:43.877 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-10-30T06:11:44.173 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2023-10-30T06:11:44.295 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2023-10-30T06:11:44.685 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-10-30T06:11:44.697 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - AppStream 17 MB/s | 6.3 MB 00:00 2023-10-30T06:11:46.130 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2023-10-30T06:11:46.142 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2023-10-30T06:11:48.074 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 7.9 MB 00:01 2023-10-30T06:11:48.207 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 7.9 MB 00:01 2023-10-30T06:11:50.459 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 13 MB/s | 18 MB 00:01 2023-10-30T06:11:51.180 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 18 MB 00:01 2023-10-30T06:11:56.252 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 5.7 MB 00:02 2023-10-30T06:11:57.255 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 5.7 MB 00:02 2023-10-30T06:11:57.644 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 15 kB 00:00 2023-10-30T06:11:58.639 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 15 kB 00:00 2023-10-30T06:11:59.056 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 15 MB/s | 19 MB 00:01 2023-10-30T06:11:59.949 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 16 MB/s | 19 MB 00:01 2023-10-30T06:12:03.612 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2023-10-30T06:12:04.432 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2023-10-30T06:12:05.979 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:12:06.032 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:12:06.033 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:06.033 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:12:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 8.7 M 2023-10-30T06:12:06.034 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2023-10-30T06:12:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.3 M 2023-10-30T06:12:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.0 M 2023-10-30T06:12:06.034 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:12:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options 2023-10-30T06:12:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-30T06:12:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.2 M 2023-10-30T06:12:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 19 M 2023-10-30T06:12:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 24 k 2023-10-30T06:12:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-30T06:12:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-30T06:12:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-30T06:12:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-30T06:12:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 715 k 2023-10-30T06:12:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-30T06:12:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-30T06:12:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-30T06:12:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 473 k 2023-10-30T06:12:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-30T06:12:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 4.8 M 2023-10-30T06:12:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-30T06:12:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-30T06:12:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-30T06:12:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse 2023-10-30T06:12:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 45 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 127 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 162 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 320 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 299 k 2023-10-30T06:12:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 99 k 2023-10-30T06:12:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-30T06:12:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-30T06:12:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:06.040 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:12:06.040 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:06.041 INFO:teuthology.orchestra.run.smithi112.stdout:Install 28 Packages 2023-10-30T06:12:06.041 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 2 Packages 2023-10-30T06:12:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:06.042 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 55 M 2023-10-30T06:12:06.042 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:12:06.798 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:12:06.801 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:06.801 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:12:06.801 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:06.801 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:12:06.801 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 8.7 M 2023-10-30T06:12:06.802 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2023-10-30T06:12:06.802 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.3 M 2023-10-30T06:12:06.802 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.0 M 2023-10-30T06:12:06.802 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:12:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options 2023-10-30T06:12:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-30T06:12:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.2 M 2023-10-30T06:12:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 19 M 2023-10-30T06:12:06.803 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 24 k 2023-10-30T06:12:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-30T06:12:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-30T06:12:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-30T06:12:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-30T06:12:06.804 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 715 k 2023-10-30T06:12:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-30T06:12:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-30T06:12:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-30T06:12:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 473 k 2023-10-30T06:12:06.805 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-30T06:12:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 4.8 M 2023-10-30T06:12:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-30T06:12:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-30T06:12:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-30T06:12:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse 2023-10-30T06:12:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 45 k 2023-10-30T06:12:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 127 k 2023-10-30T06:12:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 162 k 2023-10-30T06:12:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-30T06:12:06.807 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 320 k 2023-10-30T06:12:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 299 k 2023-10-30T06:12:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 99 k 2023-10-30T06:12:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-30T06:12:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-30T06:12:06.808 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:06.809 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:12:06.809 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:06.809 INFO:teuthology.orchestra.run.smithi079.stdout:Install 28 Packages 2023-10-30T06:12:06.809 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 2 Packages 2023-10-30T06:12:06.809 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:06.810 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 55 M 2023-10-30T06:12:06.810 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:12:07.472 INFO:teuthology.orchestra.run.smithi112.stdout:(1/30): ceph-base-18.0.0-6858.gd0db9f2f.el9.x86 5.2 MB/s | 5.2 MB 00:00 2023-10-30T06:12:07.572 INFO:teuthology.orchestra.run.smithi112.stdout:(2/30): ceph-selinux-18.0.0-6858.gd0db9f2f.el9. 242 kB/s | 24 kB 00:00 2023-10-30T06:12:07.789 INFO:teuthology.orchestra.run.smithi112.stdout:(3/30): ceph-radosgw-18.0.0-6858.gd0db9f2f.el9. 6.6 MB/s | 8.7 MB 00:01 2023-10-30T06:12:07.831 INFO:teuthology.orchestra.run.smithi112.stdout:(4/30): libcephfs2-18.0.0-6858.gd0db9f2f.el9.x8 2.7 MB/s | 715 kB 00:00 2023-10-30T06:12:07.881 INFO:teuthology.orchestra.run.smithi112.stdout:(5/30): libradosstriper1-18.0.0-6858.gd0db9f2f. 5.0 MB/s | 473 kB 00:00 2023-10-30T06:12:08.023 INFO:teuthology.orchestra.run.smithi112.stdout:(6/30): python3-ceph-argparse-18.0.0-6858.gd0db 318 kB/s | 45 kB 00:00 2023-10-30T06:12:08.124 INFO:teuthology.orchestra.run.smithi112.stdout:(7/30): python3-ceph-common-18.0.0-6858.gd0db9f 1.2 MB/s | 127 kB 00:00 2023-10-30T06:12:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:(8/30): python3-cephfs-18.0.0-6858.gd0db9f2f.el 1.6 MB/s | 162 kB 00:00 2023-10-30T06:12:08.308 INFO:teuthology.orchestra.run.smithi112.stdout:(9/30): python3-rados-18.0.0-6858.gd0db9f2f.el9 3.8 MB/s | 320 kB 00:00 2023-10-30T06:12:08.400 INFO:teuthology.orchestra.run.smithi112.stdout:(10/30): python3-rbd-18.0.0-6858.gd0db9f2f.el9. 3.2 MB/s | 299 kB 00:00 2023-10-30T06:12:08.500 INFO:teuthology.orchestra.run.smithi112.stdout:(11/30): python3-rgw-18.0.0-6858.gd0db9f2f.el9. 992 kB/s | 99 kB 00:00 2023-10-30T06:12:08.567 INFO:teuthology.orchestra.run.smithi112.stdout:(12/30): librgw2-18.0.0-6858.gd0db9f2f.el9.x86_ 6.5 MB/s | 4.8 MB 00:00 2023-10-30T06:12:08.659 INFO:teuthology.orchestra.run.smithi079.stdout:(1/30): ceph-base-18.0.0-6858.gd0db9f2f.el9.x86 3.0 MB/s | 5.2 MB 00:01 2023-10-30T06:12:08.684 INFO:teuthology.orchestra.run.smithi112.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 577 kB/s | 106 kB 00:00 2023-10-30T06:12:08.717 INFO:teuthology.orchestra.run.smithi079.stdout:(2/30): ceph-selinux-18.0.0-6858.gd0db9f2f.el9. 413 kB/s | 24 kB 00:00 2023-10-30T06:12:08.718 INFO:teuthology.orchestra.run.smithi112.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2023-10-30T06:12:08.759 INFO:teuthology.orchestra.run.smithi112.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2023-10-30T06:12:08.785 INFO:teuthology.orchestra.run.smithi112.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-10-30T06:12:08.801 INFO:teuthology.orchestra.run.smithi112.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 993 kB/s | 41 kB 00:00 2023-10-30T06:12:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:(3/30): libcephfs2-18.0.0-6858.gd0db9f2f.el9.x8 4.4 MB/s | 715 kB 00:00 2023-10-30T06:12:08.985 INFO:teuthology.orchestra.run.smithi112.stdout:(18/30): ceph-common-18.0.0-6858.gd0db9f2f.el9. 7.6 MB/s | 19 MB 00:02 2023-10-30T06:12:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:(4/30): libradosstriper1-18.0.0-6858.gd0db9f2f. 3.5 MB/s | 473 kB 00:00 2023-10-30T06:12:09.327 INFO:teuthology.orchestra.run.smithi112.stdout:(19/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 83 kB/s | 45 kB 00:00 2023-10-30T06:12:09.351 INFO:teuthology.orchestra.run.smithi079.stdout:(5/30): ceph-radosgw-18.0.0-6858.gd0db9f2f.el9. 3.6 MB/s | 8.7 MB 00:02 2023-10-30T06:12:09.460 INFO:teuthology.orchestra.run.smithi079.stdout:(6/30): python3-ceph-argparse-18.0.0-6858.gd0db 418 kB/s | 45 kB 00:00 2023-10-30T06:12:09.560 INFO:teuthology.orchestra.run.smithi079.stdout:(7/30): python3-ceph-common-18.0.0-6858.gd0db9f 1.2 MB/s | 127 kB 00:00 2023-10-30T06:12:09.652 INFO:teuthology.orchestra.run.smithi079.stdout:(8/30): python3-cephfs-18.0.0-6858.gd0db9f2f.el 1.7 MB/s | 162 kB 00:00 2023-10-30T06:12:09.702 INFO:teuthology.orchestra.run.smithi112.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 123 kB/s | 111 kB 00:00 2023-10-30T06:12:09.786 INFO:teuthology.orchestra.run.smithi079.stdout:(9/30): python3-rados-18.0.0-6858.gd0db9f2f.el9 2.3 MB/s | 320 kB 00:00 2023-10-30T06:12:09.786 INFO:teuthology.orchestra.run.smithi112.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 399 kB/s | 309 kB 00:00 2023-10-30T06:12:09.803 INFO:teuthology.orchestra.run.smithi112.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 243 kB/s | 24 kB 00:00 2023-10-30T06:12:09.890 INFO:teuthology.orchestra.run.smithi112.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2023-10-30T06:12:09.894 INFO:teuthology.orchestra.run.smithi079.stdout:(10/30): librgw2-18.0.0-6858.gd0db9f2f.el9.x86_ 5.4 MB/s | 4.8 MB 00:00 2023-10-30T06:12:09.919 INFO:teuthology.orchestra.run.smithi079.stdout:(11/30): python3-rbd-18.0.0-6858.gd0db9f2f.el9. 2.2 MB/s | 299 kB 00:00 2023-10-30T06:12:09.978 INFO:teuthology.orchestra.run.smithi079.stdout:(12/30): python3-rgw-18.0.0-6858.gd0db9f2f.el9. 1.2 MB/s | 99 kB 00:00 2023-10-30T06:12:09.986 INFO:teuthology.orchestra.run.smithi112.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2023-10-30T06:12:10.037 INFO:teuthology.orchestra.run.smithi079.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 905 kB/s | 106 kB 00:00 2023-10-30T06:12:10.062 INFO:teuthology.orchestra.run.smithi079.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2023-10-30T06:12:10.254 INFO:teuthology.orchestra.run.smithi112.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 719 kB/s | 192 kB 00:00 2023-10-30T06:12:10.254 INFO:teuthology.orchestra.run.smithi079.stdout:(15/30): ceph-common-18.0.0-6858.gd0db9f2f.el9. 5.7 MB/s | 19 MB 00:03 2023-10-30T06:12:10.279 INFO:teuthology.orchestra.run.smithi079.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 661 kB 00:00 2023-10-30T06:12:10.312 INFO:teuthology.orchestra.run.smithi079.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 282 kB 00:00 2023-10-30T06:12:10.330 INFO:teuthology.orchestra.run.smithi079.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 553 kB/s | 41 kB 00:00 2023-10-30T06:12:10.455 INFO:teuthology.orchestra.run.smithi079.stdout:(19/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 318 kB/s | 45 kB 00:00 2023-10-30T06:12:10.522 INFO:teuthology.orchestra.run.smithi079.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 530 kB/s | 111 kB 00:00 2023-10-30T06:12:10.580 INFO:teuthology.orchestra.run.smithi079.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2023-10-30T06:12:10.606 INFO:teuthology.orchestra.run.smithi079.stdout:(22/30): libarrow-doc-9.0.0-9.el9.noarch.rpm 293 kB/s | 24 kB 00:00 2023-10-30T06:12:10.656 INFO:teuthology.orchestra.run.smithi079.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2023-10-30T06:12:10.715 INFO:teuthology.orchestra.run.smithi079.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 626 kB/s | 67 kB 00:00 2023-10-30T06:12:10.790 INFO:teuthology.orchestra.run.smithi079.stdout:(25/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 6.1 MB/s | 837 kB 00:00 2023-10-30T06:12:10.866 INFO:teuthology.orchestra.run.smithi079.stdout:(26/30): libarrow-9.0.0-9.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2023-10-30T06:12:10.899 INFO:teuthology.orchestra.run.smithi079.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2023-10-30T06:12:10.954 INFO:teuthology.orchestra.run.smithi112.stdout:(26/30): parquet-libs-9.0.0-9.el9.x86_64.rpm 785 kB/s | 837 kB 00:01 2023-10-30T06:12:10.966 INFO:teuthology.orchestra.run.smithi079.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2023-10-30T06:12:11.524 INFO:teuthology.orchestra.run.smithi079.stdout:(29/30): librbd1-18.0.0-6858.gd0db9f2f.el9.x86_ 4.9 MB/s | 3.0 MB 00:00 2023-10-30T06:12:11.587 INFO:teuthology.orchestra.run.smithi112.stdout:(27/30): librados2-18.0.0-6858.gd0db9f2f.el9.x8 5.2 MB/s | 3.3 MB 00:00 2023-10-30T06:12:11.796 INFO:teuthology.orchestra.run.smithi112.stdout:(28/30): librbd1-18.0.0-6858.gd0db9f2f.el9.x86_ 15 MB/s | 3.0 MB 00:00 2023-10-30T06:12:11.808 INFO:teuthology.orchestra.run.smithi079.stdout:(30/30): librados2-18.0.0-6858.gd0db9f2f.el9.x8 3.5 MB/s | 3.3 MB 00:00 2023-10-30T06:12:11.812 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:11.813 INFO:teuthology.orchestra.run.smithi079.stdout:Total 11 MB/s | 55 MB 00:05 2023-10-30T06:12:11.823 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:12:11.858 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:12:11.858 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:12:12.263 INFO:teuthology.orchestra.run.smithi112.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 806 kB/s | 1.6 MB 00:02 2023-10-30T06:12:12.334 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:12:12.335 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:12:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:12:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-30T06:12:13.867 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-10-30T06:12:13.921 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-10-30T06:12:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-10-30T06:12:14.303 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:14.394 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:14.489 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/32 2023-10-30T06:12:14.514 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:14.560 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:14.598 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-10-30T06:12:14.640 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9. 9/32 2023-10-30T06:12:14.707 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 10/32 2023-10-30T06:12:14.733 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 11/32 2023-10-30T06:12:14.781 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 11/32 2023-10-30T06:12:14.886 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-30T06:12:14.922 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-30T06:12:14.963 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-30T06:12:15.003 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-30T06:12:15.031 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-30T06:12:15.066 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-30T06:12:15.099 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-30T06:12:15.229 INFO:teuthology.orchestra.run.smithi112.stdout:(30/30): libarrow-9.0.0-9.el9.x86_64.rpm 766 kB/s | 4.4 MB 00:05 2023-10-30T06:12:15.236 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:15.237 INFO:teuthology.orchestra.run.smithi112.stdout:Total 6.0 MB/s | 55 MB 00:09 2023-10-30T06:12:15.249 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:12:15.284 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:12:15.284 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:12:15.402 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-30T06:12:15.429 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 20/32 2023-10-30T06:12:15.510 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 20/32 2023-10-30T06:12:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 21/32 2023-10-30T06:12:15.586 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-30T06:12:15.749 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:12:15.749 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:12:16.030 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-30T06:12:16.065 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 24/32 2023-10-30T06:12:16.120 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 24/32 2023-10-30T06:12:16.196 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 25/32 2023-10-30T06:12:16.226 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x8 26/32 2023-10-30T06:12:17.142 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:12:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2023-10-30T06:12:17.283 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/32 2023-10-30T06:12:17.335 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/32 2023-10-30T06:12:17.646 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2023-10-30T06:12:17.708 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:17.782 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:17.868 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/32 2023-10-30T06:12:17.910 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:17.956 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2023-10-30T06:12:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9. 9/32 2023-10-30T06:12:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 10/32 2023-10-30T06:12:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 11/32 2023-10-30T06:12:18.219 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 11/32 2023-10-30T06:12:18.325 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2023-10-30T06:12:18.353 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/32 2023-10-30T06:12:18.392 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2023-10-30T06:12:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:18.425 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2023-10-30T06:12:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:18.452 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/32 2023-10-30T06:12:18.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2023-10-30T06:12:18.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-30T06:12:18.822 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2023-10-30T06:12:18.850 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 20/32 2023-10-30T06:12:18.923 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 20/32 2023-10-30T06:12:18.955 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 21/32 2023-10-30T06:12:18.991 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2023-10-30T06:12:19.002 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:19.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 28/32 2023-10-30T06:12:19.131 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 28/32 2023-10-30T06:12:19.131 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-30T06:12:19.132 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-30T06:12:19.132 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:19.163 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:19.418 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 23/32 2023-10-30T06:12:19.445 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 24/32 2023-10-30T06:12:19.517 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 24/32 2023-10-30T06:12:19.583 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 25/32 2023-10-30T06:12:19.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x8 26/32 2023-10-30T06:12:21.808 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:21.844 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 27/32 2023-10-30T06:12:22.449 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 28/32 2023-10-30T06:12:22.519 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 28/32 2023-10-30T06:12:22.520 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-30T06:12:22.520 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-30T06:12:22.520 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:22.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:28.431 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:28.432 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2023-10-30T06:12:28.432 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2023-10-30T06:12:28.432 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2023-10-30T06:12:28.432 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2023-10-30T06:12:28.433 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2023-10-30T06:12:28.433 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2023-10-30T06:12:28.433 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2023-10-30T06:12:28.433 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:29.316 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/32 2023-10-30T06:12:29.340 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/32 2023-10-30T06:12:29.340 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:29.341 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-30T06:12:29.341 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-30T06:12:29.341 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-30T06:12:29.341 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:29.342 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-30T06:12:29.436 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-30T06:12:29.437 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:30.653 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:30.653 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/32 2023-10-30T06:12:30.653 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/32 2023-10-30T06:12:30.654 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 3/32 2023-10-30T06:12:30.654 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 4/32 2023-10-30T06:12:30.654 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:30.654 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 6/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9. 8/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x8 9/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 10/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 11/32 2023-10-30T06:12:30.655 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 12/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 13/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-30T06:12:30.656 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-30T06:12:30.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-30T06:12:30.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-30T06:12:30.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-30T06:12:30.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-30T06:12:30.657 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-30T06:12:30.658 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-30T06:12:30.658 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-30T06:12:30.658 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-30T06:12:30.658 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-30T06:12:30.659 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-30T06:12:30.659 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:30.659 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-30T06:12:30.659 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/32 2023-10-30T06:12:31.641 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:31.641 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2023-10-30T06:12:31.642 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2023-10-30T06:12:31.642 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2023-10-30T06:12:31.642 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2023-10-30T06:12:31.642 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2023-10-30T06:12:31.643 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2023-10-30T06:12:31.643 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2023-10-30T06:12:31.643 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:31.690 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:31.691 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2023-10-30T06:12:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.691 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:12:31.691 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-30T06:12:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.692 INFO:teuthology.orchestra.run.smithi079.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-30T06:12:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-30T06:12:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-30T06:12:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-30T06:12:31.693 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-30T06:12:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-30T06:12:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-30T06:12:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.694 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-30T06:12:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-30T06:12:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-30T06:12:31.695 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-30T06:12:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-30T06:12:31.696 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2023-10-30T06:12:31.697 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-30T06:12:31.698 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:31.698 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:12:32.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-test 2023-10-30T06:12:32.474 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/32 2023-10-30T06:12:32.499 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/32 2023-10-30T06:12:32.499 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:32.500 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-30T06:12:32.500 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-30T06:12:32.500 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-30T06:12:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-30T06:12:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2023-10-30T06:12:32.569 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:32.581 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:28 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:12:32.721 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:12:32.722 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:32.722 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:12:32.723 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:32.723 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:12:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 41 M 2023-10-30T06:12:32.723 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:12:32.723 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-30T06:12:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 164 k 2023-10-30T06:12:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-30T06:12:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-30T06:12:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-30T06:12:32.724 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:32.725 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:12:32.725 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:12:32.725 INFO:teuthology.orchestra.run.smithi079.stdout:Install 6 Packages 2023-10-30T06:12:32.725 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:32.726 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 42 M 2023-10-30T06:12:32.726 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 172 M 2023-10-30T06:12:32.726 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:12:33.291 INFO:teuthology.orchestra.run.smithi079.stdout:(1/6): libcephsqlite-18.0.0-6858.gd0db9f2f.el9. 643 kB/s | 164 kB 00:00 2023-10-30T06:12:33.325 INFO:teuthology.orchestra.run.smithi079.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 753 kB/s | 217 kB 00:00 2023-10-30T06:12:33.392 INFO:teuthology.orchestra.run.smithi079.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.9 MB/s | 300 kB 00:00 2023-10-30T06:12:33.417 INFO:teuthology.orchestra.run.smithi079.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 680 kB/s | 63 kB 00:00 2023-10-30T06:12:33.642 INFO:teuthology.orchestra.run.smithi079.stdout:(5/6): jq-1.6-15.el9.x86_64.rpm 747 kB/s | 187 kB 00:00 2023-10-30T06:12:33.731 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:33.732 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/32 2023-10-30T06:12:33.732 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/32 2023-10-30T06:12:33.732 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 3/32 2023-10-30T06:12:33.732 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 4/32 2023-10-30T06:12:33.733 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/32 2023-10-30T06:12:33.733 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_6 6/32 2023-10-30T06:12:33.733 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 7/32 2023-10-30T06:12:33.733 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9. 8/32 2023-10-30T06:12:33.734 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x8 9/32 2023-10-30T06:12:33.734 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 10/32 2023-10-30T06:12:33.734 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 11/32 2023-10-30T06:12:33.734 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 12/32 2023-10-30T06:12:33.734 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 13/32 2023-10-30T06:12:33.735 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2023-10-30T06:12:33.735 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2023-10-30T06:12:33.735 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/32 2023-10-30T06:12:33.735 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/32 2023-10-30T06:12:33.735 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/32 2023-10-30T06:12:33.736 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/32 2023-10-30T06:12:33.736 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2023-10-30T06:12:33.736 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2023-10-30T06:12:33.736 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 22/32 2023-10-30T06:12:33.736 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 23/32 2023-10-30T06:12:33.737 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2023-10-30T06:12:33.737 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2023-10-30T06:12:33.737 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 26/32 2023-10-30T06:12:33.737 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2023-10-30T06:12:33.737 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2023-10-30T06:12:33.738 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/32 2023-10-30T06:12:33.738 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2023-10-30T06:12:33.738 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/32 2023-10-30T06:12:34.785 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2023-10-30T06:12:34.786 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:34.786 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2023-10-30T06:12:34.786 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.786 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.787 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:12:34.787 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-30T06:12:34.787 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.787 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.788 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.788 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.788 INFO:teuthology.orchestra.run.smithi112.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-30T06:12:34.788 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-30T06:12:34.788 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-30T06:12:34.789 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-30T06:12:34.789 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.789 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-30T06:12:34.789 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-30T06:12:34.790 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-30T06:12:34.790 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.790 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-30T06:12:34.790 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.790 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-30T06:12:34.791 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-30T06:12:34.791 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-30T06:12:34.791 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.791 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.791 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.792 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-30T06:12:34.792 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.792 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.792 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:34.792 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2023-10-30T06:12:34.793 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-30T06:12:34.793 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:34.793 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:12:35.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-test 2023-10-30T06:12:35.601 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:32 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:12:35.738 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:12:35.739 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:35.739 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:12:35.740 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:35.740 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:12:35.740 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 41 M 2023-10-30T06:12:35.740 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:12:35.740 INFO:teuthology.orchestra.run.smithi112.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-30T06:12:35.741 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 164 k 2023-10-30T06:12:35.741 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-30T06:12:35.741 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-30T06:12:35.741 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-30T06:12:35.741 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:35.742 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:12:35.742 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:12:35.742 INFO:teuthology.orchestra.run.smithi112.stdout:Install 6 Packages 2023-10-30T06:12:35.742 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:35.743 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 42 M 2023-10-30T06:12:35.743 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 172 M 2023-10-30T06:12:35.743 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:12:36.284 INFO:teuthology.orchestra.run.smithi112.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.0 MB/s | 217 kB 00:00 2023-10-30T06:12:36.358 INFO:teuthology.orchestra.run.smithi112.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2023-10-30T06:12:36.409 INFO:teuthology.orchestra.run.smithi112.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-10-30T06:12:36.817 INFO:teuthology.orchestra.run.smithi112.stdout:(4/6): libcephsqlite-18.0.0-6858.gd0db9f2f.el9. 218 kB/s | 164 kB 00:00 2023-10-30T06:12:37.009 INFO:teuthology.orchestra.run.smithi112.stdout:(5/6): jq-1.6-15.el9.x86_64.rpm 311 kB/s | 187 kB 00:00 2023-10-30T06:12:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:(6/6): ceph-test-18.0.0-6858.gd0db9f2f.el9.x86_ 9.4 MB/s | 41 MB 00:04 2023-10-30T06:12:37.395 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:37.396 INFO:teuthology.orchestra.run.smithi079.stdout:Total 8.9 MB/s | 42 MB 00:04 2023-10-30T06:12:37.409 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:12:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:12:37.431 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:12:37.802 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:12:37.802 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:12:38.342 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:12:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-10-30T06:12:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-10-30T06:12:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-30T06:12:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-10-30T06:12:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/6 2023-10-30T06:12:39.109 INFO:teuthology.orchestra.run.smithi112.stdout:(6/6): ceph-test-18.0.0-6858.gd0db9f2f.el9.x86_ 13 MB/s | 41 MB 00:03 2023-10-30T06:12:39.114 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:39.114 INFO:teuthology.orchestra.run.smithi112.stdout:Total 12 MB/s | 42 MB 00:03 2023-10-30T06:12:39.127 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:12:39.147 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:12:39.148 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:12:39.511 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:12:39.512 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:12:40.048 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:12:40.131 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-10-30T06:12:40.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-10-30T06:12:40.209 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-30T06:12:40.268 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : jq-1.6-15.el9.x86_64 4/6 2023-10-30T06:12:40.296 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/6 2023-10-30T06:12:42.448 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/6 2023-10-30T06:12:42.529 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/6 2023-10-30T06:12:44.338 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/6 2023-10-30T06:12:44.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/6 2023-10-30T06:12:44.716 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/6 2023-10-30T06:12:44.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/6 2023-10-30T06:12:44.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/6 2023-10-30T06:12:44.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-30T06:12:44.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-10-30T06:12:44.717 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-10-30T06:12:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-10-30T06:12:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:45.115 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:12:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.6-15.el9.x86_64 2023-10-30T06:12:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-30T06:12:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-30T06:12:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-30T06:12:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:45.117 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:12:45.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph 2023-10-30T06:12:45.959 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:41 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:12:46.103 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:12:46.105 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================= 2023-10-30T06:12:46.105 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:12:46.105 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================= 2023-10-30T06:12:46.105 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:12:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 6.5 k 2023-10-30T06:12:46.106 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:12:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 2.1 M 2023-10-30T06:12:46.106 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.5 M 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 243 k 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 4.4 M 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 18 M 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-30T06:12:46.107 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-30T06:12:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-30T06:12:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-30T06:12:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-30T06:12:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-30T06:12:46.108 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-30T06:12:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-30T06:12:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-30T06:12:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-30T06:12:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-30T06:12:46.109 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-30T06:12:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-30T06:12:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-30T06:12:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-30T06:12:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-30T06:12:46.110 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-30T06:12:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-30T06:12:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-30T06:12:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-30T06:12:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-30T06:12:46.111 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-30T06:12:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-30T06:12:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-30T06:12:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-30T06:12:46.112 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-30T06:12:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:46.113 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:12:46.113 INFO:teuthology.orchestra.run.smithi079.stdout:========================================================================================= 2023-10-30T06:12:46.113 INFO:teuthology.orchestra.run.smithi079.stdout:Install 33 Packages 2023-10-30T06:12:46.113 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:46.114 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 30 M 2023-10-30T06:12:46.114 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 110 M 2023-10-30T06:12:46.114 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:12:46.503 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 6/6 2023-10-30T06:12:46.503 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/6 2023-10-30T06:12:46.503 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/6 2023-10-30T06:12:46.504 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-10-30T06:12:46.504 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2023-10-30T06:12:46.504 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2023-10-30T06:12:46.905 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : jq-1.6-15.el9.x86_64 6/6 2023-10-30T06:12:46.905 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:46.906 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:12:46.906 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:46.906 INFO:teuthology.orchestra.run.smithi112.stdout: jq-1.6-15.el9.x86_64 2023-10-30T06:12:46.906 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:12:46.907 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-30T06:12:46.907 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-30T06:12:46.907 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-30T06:12:46.907 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:46.907 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:12:47.189 INFO:teuthology.orchestra.run.smithi079.stdout:(1/33): ceph-18.0.0-6858.gd0db9f2f.el9.x86_64.r 18 kB/s | 6.5 kB 00:00 2023-10-30T06:12:47.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph 2023-10-30T06:12:47.745 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:44 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:12:47.886 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:12:47.888 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================= 2023-10-30T06:12:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:12:47.888 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================= 2023-10-30T06:12:47.889 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:12:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 6.5 k 2023-10-30T06:12:47.889 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:12:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 2.1 M 2023-10-30T06:12:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.5 M 2023-10-30T06:12:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 243 k 2023-10-30T06:12:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 4.4 M 2023-10-30T06:12:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 18 M 2023-10-30T06:12:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-30T06:12:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-30T06:12:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-30T06:12:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-30T06:12:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-30T06:12:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-30T06:12:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-30T06:12:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-30T06:12:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-30T06:12:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-30T06:12:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-30T06:12:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-30T06:12:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-30T06:12:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-30T06:12:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-30T06:12:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-30T06:12:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-30T06:12:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-30T06:12:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-30T06:12:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-30T06:12:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-30T06:12:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-30T06:12:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-30T06:12:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-30T06:12:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-30T06:12:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-30T06:12:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-30T06:12:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:47.896 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:12:47.896 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================= 2023-10-30T06:12:47.897 INFO:teuthology.orchestra.run.smithi112.stdout:Install 33 Packages 2023-10-30T06:12:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:47.897 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 30 M 2023-10-30T06:12:47.898 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 110 M 2023-10-30T06:12:47.898 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:12:48.356 INFO:teuthology.orchestra.run.smithi079.stdout:(2/33): ceph-mgr-18.0.0-6858.gd0db9f2f.el9.x86_ 974 kB/s | 1.5 MB 00:01 2023-10-30T06:12:49.014 INFO:teuthology.orchestra.run.smithi079.stdout:(3/33): ceph-mds-18.0.0-6858.gd0db9f2f.el9.x86_ 973 kB/s | 2.1 MB 00:02 2023-10-30T06:12:49.019 INFO:teuthology.orchestra.run.smithi112.stdout:(1/33): ceph-18.0.0-6858.gd0db9f2f.el9.x86_64.r 18 kB/s | 6.5 kB 00:00 2023-10-30T06:12:49.215 INFO:teuthology.orchestra.run.smithi079.stdout:(4/33): ceph-mgr-modules-core-18.0.0-6858.gd0db 1.2 MB/s | 243 kB 00:00 2023-10-30T06:12:49.349 INFO:teuthology.orchestra.run.smithi079.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-10-30T06:12:49.391 INFO:teuthology.orchestra.run.smithi079.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 759 kB/s | 32 kB 00:00 2023-10-30T06:12:49.441 INFO:teuthology.orchestra.run.smithi079.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2023-10-30T06:12:49.491 INFO:teuthology.orchestra.run.smithi079.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-10-30T06:12:49.533 INFO:teuthology.orchestra.run.smithi079.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2023-10-30T06:12:49.576 INFO:teuthology.orchestra.run.smithi079.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2023-10-30T06:12:49.617 INFO:teuthology.orchestra.run.smithi079.stdout:(11/33): python3-requests-2.25.1-7.el9.noarch.r 2.6 MB/s | 113 kB 00:00 2023-10-30T06:12:49.659 INFO:teuthology.orchestra.run.smithi079.stdout:(12/33): python3-urllib3-1.26.5-4.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2023-10-30T06:12:50.085 INFO:teuthology.orchestra.run.smithi079.stdout:(13/33): ceph-mon-18.0.0-6858.gd0db9f2f.el9.x86 1.5 MB/s | 4.4 MB 00:02 2023-10-30T06:12:50.185 INFO:teuthology.orchestra.run.smithi079.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 511 kB/s | 51 kB 00:00 2023-10-30T06:12:50.286 INFO:teuthology.orchestra.run.smithi112.stdout:(2/33): ceph-mds-18.0.0-6858.gd0db9f2f.el9.x86_ 1.3 MB/s | 2.1 MB 00:01 2023-10-30T06:12:50.294 INFO:teuthology.orchestra.run.smithi079.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 402 kB/s | 43 kB 00:00 2023-10-30T06:12:50.353 INFO:teuthology.orchestra.run.smithi079.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2023-10-30T06:12:50.361 INFO:teuthology.orchestra.run.smithi112.stdout:(3/33): ceph-mgr-18.0.0-6858.gd0db9f2f.el9.x86_ 878 kB/s | 1.5 MB 00:01 2023-10-30T06:12:50.403 INFO:teuthology.orchestra.run.smithi079.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2023-10-30T06:12:50.436 INFO:teuthology.orchestra.run.smithi079.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 321 kB/s | 11 kB 00:00 2023-10-30T06:12:50.470 INFO:teuthology.orchestra.run.smithi079.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 529 kB/s | 18 kB 00:00 2023-10-30T06:12:50.478 INFO:teuthology.orchestra.run.smithi112.stdout:(4/33): ceph-mgr-modules-core-18.0.0-6858.gd0db 2.0 MB/s | 243 kB 00:00 2023-10-30T06:12:50.504 INFO:teuthology.orchestra.run.smithi079.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 694 kB/s | 23 kB 00:00 2023-10-30T06:12:50.538 INFO:teuthology.orchestra.run.smithi079.stdout:(21/33): python3-cryptography-36.0.1-4.el9.x86_ 1.4 MB/s | 1.2 MB 00:00 2023-10-30T06:12:50.571 INFO:teuthology.orchestra.run.smithi079.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 290 kB/s | 19 kB 00:00 2023-10-30T06:12:50.587 INFO:teuthology.orchestra.run.smithi112.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2023-10-30T06:12:50.597 INFO:teuthology.orchestra.run.smithi079.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2023-10-30T06:12:50.620 INFO:teuthology.orchestra.run.smithi112.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 955 kB/s | 32 kB 00:00 2023-10-30T06:12:50.622 INFO:teuthology.orchestra.run.smithi079.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2023-10-30T06:12:50.647 INFO:teuthology.orchestra.run.smithi079.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2023-10-30T06:12:50.654 INFO:teuthology.orchestra.run.smithi112.stdout:(7/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2023-10-30T06:12:50.672 INFO:teuthology.orchestra.run.smithi079.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2023-10-30T06:12:50.689 INFO:teuthology.orchestra.run.smithi079.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 397 kB/s | 16 kB 00:00 2023-10-30T06:12:50.696 INFO:teuthology.orchestra.run.smithi112.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2023-10-30T06:12:50.724 INFO:teuthology.orchestra.run.smithi079.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2023-10-30T06:12:50.740 INFO:teuthology.orchestra.run.smithi079.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 705 kB/s | 36 kB 00:00 2023-10-30T06:12:50.763 INFO:teuthology.orchestra.run.smithi112.stdout:(9/33): ceph-mon-18.0.0-6858.gd0db9f2f.el9.x86_ 2.5 MB/s | 4.4 MB 00:01 2023-10-30T06:12:50.788 INFO:teuthology.orchestra.run.smithi112.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-10-30T06:12:50.813 INFO:teuthology.orchestra.run.smithi112.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2023-10-30T06:12:50.833 INFO:teuthology.orchestra.run.smithi079.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-10-30T06:12:50.839 INFO:teuthology.orchestra.run.smithi112.stdout:(12/33): python3-requests-2.25.1-7.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2023-10-30T06:12:50.864 INFO:teuthology.orchestra.run.smithi112.stdout:(13/33): python3-urllib3-1.26.5-4.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2023-10-30T06:12:50.866 INFO:teuthology.orchestra.run.smithi079.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2023-10-30T06:12:50.892 INFO:teuthology.orchestra.run.smithi079.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 346 kB/s | 20 kB 00:00 2023-10-30T06:12:51.048 INFO:teuthology.orchestra.run.smithi112.stdout:(14/33): python3-cryptography-36.0.1-4.el9.x86_ 5.9 MB/s | 1.2 MB 00:00 2023-10-30T06:12:51.106 INFO:teuthology.orchestra.run.smithi112.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 748 kB/s | 43 kB 00:00 2023-10-30T06:12:51.156 INFO:teuthology.orchestra.run.smithi112.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2023-10-30T06:12:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2023-10-30T06:12:51.232 INFO:teuthology.orchestra.run.smithi112.stdout:(18/33): python3-pytz-2021.1-5.el9.noarch.rpm 139 kB/s | 51 kB 00:00 2023-10-30T06:12:51.249 INFO:teuthology.orchestra.run.smithi112.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 255 kB/s | 11 kB 00:00 2023-10-30T06:12:51.274 INFO:teuthology.orchestra.run.smithi112.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 421 kB/s | 18 kB 00:00 2023-10-30T06:12:51.299 INFO:teuthology.orchestra.run.smithi112.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 785 kB/s | 19 kB 00:00 2023-10-30T06:12:51.325 INFO:teuthology.orchestra.run.smithi112.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2023-10-30T06:12:51.342 INFO:teuthology.orchestra.run.smithi112.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 470 kB/s | 20 kB 00:00 2023-10-30T06:12:51.358 INFO:teuthology.orchestra.run.smithi112.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2023-10-30T06:12:51.384 INFO:teuthology.orchestra.run.smithi112.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2023-10-30T06:12:51.409 INFO:teuthology.orchestra.run.smithi112.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2023-10-30T06:12:51.434 INFO:teuthology.orchestra.run.smithi112.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2023-10-30T06:12:51.460 INFO:teuthology.orchestra.run.smithi112.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2023-10-30T06:12:51.485 INFO:teuthology.orchestra.run.smithi112.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2023-10-30T06:12:51.510 INFO:teuthology.orchestra.run.smithi112.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2023-10-30T06:12:51.544 INFO:teuthology.orchestra.run.smithi112.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2023-10-30T06:12:51.569 INFO:teuthology.orchestra.run.smithi112.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2023-10-30T06:12:52.111 INFO:teuthology.orchestra.run.smithi112.stdout:(33/33): ceph-osd-18.0.0-6858.gd0db9f2f.el9.x86 9.7 MB/s | 18 MB 00:01 2023-10-30T06:12:52.117 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:52.118 INFO:teuthology.orchestra.run.smithi112.stdout:Total 7.1 MB/s | 30 MB 00:04 2023-10-30T06:12:52.148 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:12:52.186 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:12:52.186 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:12:52.283 INFO:teuthology.orchestra.run.smithi079.stdout:(33/33): ceph-osd-18.0.0-6858.gd0db9f2f.el9.x86 4.5 MB/s | 18 MB 00:03 2023-10-30T06:12:52.288 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:12:52.289 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.8 MB/s | 30 MB 00:06 2023-10-30T06:12:52.322 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:12:52.361 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:12:52.361 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:12:52.562 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:12:52.563 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:12:52.738 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:12:52.738 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:12:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:12:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-30T06:12:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-30T06:12:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-30T06:12:54.018 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:12:54.046 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-30T06:12:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2023-10-30T06:12:54.118 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2023-10-30T06:12:54.133 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-30T06:12:54.151 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2023-10-30T06:12:54.173 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-30T06:12:54.211 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-30T06:12:54.242 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2023-10-30T06:12:54.271 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-30T06:12:54.321 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2023-10-30T06:12:54.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 9/33 2023-10-30T06:12:54.362 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-30T06:12:54.363 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2023-10-30T06:12:54.408 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2023-10-30T06:12:54.411 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-30T06:12:54.473 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-30T06:12:54.476 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 8/33 2023-10-30T06:12:54.531 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 9/33 2023-10-30T06:12:54.569 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 10/33 2023-10-30T06:12:54.630 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 11/33 2023-10-30T06:12:54.666 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-30T06:12:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-30T06:12:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-30T06:12:54.753 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-30T06:12:54.788 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-30T06:12:54.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-30T06:12:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 13/33 2023-10-30T06:12:54.886 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-30T06:12:54.920 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-30T06:12:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 14/33 2023-10-30T06:12:54.953 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-30T06:12:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 15/33 2023-10-30T06:12:54.994 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-30T06:12:55.009 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 16/33 2023-10-30T06:12:55.021 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-30T06:12:55.047 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 17/33 2023-10-30T06:12:55.109 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-30T06:12:55.123 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 18/33 2023-10-30T06:12:55.158 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 19/33 2023-10-30T06:12:55.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-30T06:12:55.191 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 20/33 2023-10-30T06:12:55.232 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 21/33 2023-10-30T06:12:55.240 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-30T06:12:55.259 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2023-10-30T06:12:55.347 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2023-10-30T06:12:55.377 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-30T06:12:55.390 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2023-10-30T06:12:55.461 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2023-10-30T06:12:55.501 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-30T06:12:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2023-10-30T06:12:55.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9. 28/33 2023-10-30T06:12:55.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/33 2023-10-30T06:12:55.731 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/33 2023-10-30T06:12:55.731 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:55.732 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-30T06:12:55.732 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-30T06:12:55.732 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-30T06:12:55.733 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:55.741 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-30T06:12:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9. 28/33 2023-10-30T06:12:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/33 2023-10-30T06:12:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 29/33 2023-10-30T06:12:55.945 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:55.945 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-30T06:12:55.945 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-30T06:12:55.946 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-30T06:12:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:57.308 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/33 2023-10-30T06:12:57.335 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/33 2023-10-30T06:12:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-30T06:12:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-30T06:12:57.336 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-30T06:12:57.337 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:57.506 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/33 2023-10-30T06:12:57.534 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 30/33 2023-10-30T06:12:57.534 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:57.534 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-30T06:12:57.535 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-30T06:12:57.535 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-30T06:12:57.535 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:57.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/33 2023-10-30T06:12:57.797 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/33 2023-10-30T06:12:57.798 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:57.798 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-30T06:12:57.798 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-30T06:12:57.799 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-30T06:12:57.799 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:57.974 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/33 2023-10-30T06:12:57.997 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 31/33 2023-10-30T06:12:57.998 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:57.998 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-30T06:12:57.998 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-30T06:12:57.999 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-30T06:12:57.999 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:58.001 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 32/33 2023-10-30T06:12:58.024 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 32/33 2023-10-30T06:12:58.024 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:58.024 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-30T06:12:58.024 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-30T06:12:58.025 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-30T06:12:58.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:12:58.090 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 33/33 2023-10-30T06:12:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 32/33 2023-10-30T06:12:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 32/33 2023-10-30T06:12:58.231 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:12:58.231 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-30T06:12:58.232 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-30T06:12:58.232 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-30T06:12:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:12:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 33/33 2023-10-30T06:12:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 33/33 2023-10-30T06:12:59.740 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/33 2023-10-30T06:12:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/33 2023-10-30T06:12:59.741 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 3/33 2023-10-30T06:12:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 4/33 2023-10-30T06:12:59.742 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/33 2023-10-30T06:12:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9. 6/33 2023-10-30T06:12:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-30T06:12:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-30T06:12:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-30T06:12:59.743 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 14/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-30T06:12:59.744 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-30T06:12:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-30T06:12:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-30T06:12:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-30T06:12:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-30T06:12:59.745 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-30T06:12:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-30T06:12:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-30T06:12:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-30T06:12:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-30T06:12:59.746 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-30T06:12:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-30T06:12:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-30T06:12:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-30T06:12:59.747 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-30T06:12:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-30T06:12:59.748 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-30T06:13:00.060 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 33/33 2023-10-30T06:13:00.061 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/33 2023-10-30T06:13:00.061 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/33 2023-10-30T06:13:00.061 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 3/33 2023-10-30T06:13:00.061 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 4/33 2023-10-30T06:13:00.062 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 5/33 2023-10-30T06:13:00.062 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9. 6/33 2023-10-30T06:13:00.062 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2023-10-30T06:13:00.062 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2023-10-30T06:13:00.062 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 9/33 2023-10-30T06:13:00.063 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 10/33 2023-10-30T06:13:00.063 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/33 2023-10-30T06:13:00.063 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/33 2023-10-30T06:13:00.063 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 13/33 2023-10-30T06:13:00.063 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 14/33 2023-10-30T06:13:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 15/33 2023-10-30T06:13:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/33 2023-10-30T06:13:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2023-10-30T06:13:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2023-10-30T06:13:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2023-10-30T06:13:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2023-10-30T06:13:00.066 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2023-10-30T06:13:00.066 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2023-10-30T06:13:00.066 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2023-10-30T06:13:00.067 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2023-10-30T06:13:00.067 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2023-10-30T06:13:00.067 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2023-10-30T06:13:00.067 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2023-10-30T06:13:00.672 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-30T06:13:00.673 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:00.673 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:00.673 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.673 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.673 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.674 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:00.674 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.674 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-30T06:13:00.674 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-30T06:13:00.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-30T06:13:00.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-30T06:13:00.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-30T06:13:00.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-30T06:13:00.675 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-30T06:13:00.676 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-30T06:13:00.676 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-30T06:13:00.676 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-30T06:13:00.676 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-30T06:13:00.676 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-30T06:13:00.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-30T06:13:00.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-30T06:13:00.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-30T06:13:00.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2023-10-30T06:13:00.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-30T06:13:00.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-30T06:13:00.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-30T06:13:00.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-30T06:13:00.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-30T06:13:00.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-30T06:13:00.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-30T06:13:00.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-30T06:13:00.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-30T06:13:00.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-30T06:13:00.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-30T06:13:00.680 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:00.680 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:00.976 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2023-10-30T06:13:00.977 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:00.977 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:00.977 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.977 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.977 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.978 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:00.978 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.978 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:00.978 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-30T06:13:00.978 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-30T06:13:00.979 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-30T06:13:00.979 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-30T06:13:00.979 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-30T06:13:00.979 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-30T06:13:00.979 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-30T06:13:00.980 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-30T06:13:00.980 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-30T06:13:00.980 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-30T06:13:00.980 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-30T06:13:00.980 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-30T06:13:00.981 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-30T06:13:00.981 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-30T06:13:00.981 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-30T06:13:00.981 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2023-10-30T06:13:00.981 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-30T06:13:00.982 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-30T06:13:00.982 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-30T06:13:00.982 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-30T06:13:00.982 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-30T06:13:00.982 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-30T06:13:00.983 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-30T06:13:00.983 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-30T06:13:00.983 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-30T06:13:00.983 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-30T06:13:00.983 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-30T06:13:00.984 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:00.984 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:01.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-base 2023-10-30T06:13:01.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-base 2023-10-30T06:13:01.574 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:58 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:01.665 INFO:teuthology.orchestra.run.smithi112.stdout:Package ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:01.713 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:01.714 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:01.715 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:01.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install cephadm 2023-10-30T06:13:01.938 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:57 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:02.029 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-base-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:02.078 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:02.079 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:02.079 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:02.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install cephadm 2023-10-30T06:13:02.316 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:59 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:02.454 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:02.455 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:02.455 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:02.455 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 270 k 2023-10-30T06:13:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:02.456 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:02.456 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:02.456 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 270 k 2023-10-30T06:13:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 265 k 2023-10-30T06:13:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:02.695 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:58 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:02.832 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:02.833 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:02.833 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:02.834 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:02.834 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:02.835 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 270 k 2023-10-30T06:13:02.835 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:02.835 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:02.835 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:02.835 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:02.836 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:02.836 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 270 k 2023-10-30T06:13:02.836 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 265 k 2023-10-30T06:13:02.836 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:02.974 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm-18.0.0-6858.gd0db9f2f.el9.noarch.rpm 522 kB/s | 270 kB 00:00 2023-10-30T06:13:02.974 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:02.975 INFO:teuthology.orchestra.run.smithi112.stdout:Total 521 kB/s | 270 kB 00:00 2023-10-30T06:13:02.975 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:02.979 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:02.979 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:02.982 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:02.982 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:03.377 INFO:teuthology.orchestra.run.smithi079.stdout:cephadm-18.0.0-6858.gd0db9f2f.el9.noarch.rpm 497 kB/s | 270 kB 00:00 2023-10-30T06:13:03.378 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:03.378 INFO:teuthology.orchestra.run.smithi079.stdout:Total 496 kB/s | 270 kB 00:00 2023-10-30T06:13:03.379 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:03.383 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:03.383 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:03.386 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:03.387 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:03.486 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:03.577 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:03.850 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:03.957 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:04.225 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:04.225 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:04.302 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:04.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-immutable-object-cache 2023-10-30T06:13:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:04.616 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:04.616 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:04.617 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:04.617 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:04.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-immutable-object-cache 2023-10-30T06:13:05.005 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:02 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:05.149 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:05.150 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:13:05.151 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:05.151 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 141 k 2023-10-30T06:13:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:05.152 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 141 k 2023-10-30T06:13:05.153 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 395 k 2023-10-30T06:13:05.153 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:05.377 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:01 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:05.522 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:05.523 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:05.523 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:13:05.523 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:05.523 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 141 k 2023-10-30T06:13:05.524 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:05.524 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:05.524 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:05.524 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:05.525 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:05.525 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 141 k 2023-10-30T06:13:05.525 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 395 k 2023-10-30T06:13:05.525 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:05.570 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-immutable-object-cache-18.0.0-6858.gd0db9f 337 kB/s | 141 kB 00:00 2023-10-30T06:13:05.570 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:05.571 INFO:teuthology.orchestra.run.smithi112.stdout:Total 336 kB/s | 141 kB 00:00 2023-10-30T06:13:05.571 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:05.578 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:05.578 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:05.630 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:05.631 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:05.727 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-immutable-object-cache-18.0.0-6858.gd0db9f 693 kB/s | 141 kB 00:00 2023-10-30T06:13:05.728 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:05.728 INFO:teuthology.orchestra.run.smithi079.stdout:Total 689 kB/s | 141 kB 00:00 2023-10-30T06:13:05.729 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:05.736 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:05.736 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:05.788 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:05.789 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:05.853 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:05.854 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-30T06:13:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:05.944 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:06.004 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:06.028 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:06.028 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:06.028 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-30T06:13:06.028 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:06.812 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:06.812 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:06.812 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:06.812 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:06.813 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:06.813 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:07.017 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f. 1/1 2023-10-30T06:13:07.018 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:07.018 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:07.018 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:07.018 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:07.019 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:07.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr 2023-10-30T06:13:07.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr 2023-10-30T06:13:07.608 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:04 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:07.696 INFO:teuthology.orchestra.run.smithi112.stdout:Package ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:07.745 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:07.746 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:07.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-dashboard 2023-10-30T06:13:07.862 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:03 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:07.955 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-mgr-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:08.003 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:08.004 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:08.005 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:08.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-dashboard 2023-10-30T06:13:08.341 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:05 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:08.479 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:08.480 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:08.480 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:08.480 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:08.481 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:08.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 1.8 M 2023-10-30T06:13:08.481 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:08.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 23 k 2023-10-30T06:13:08.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 15 k 2023-10-30T06:13:08.482 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-30T06:13:08.482 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-30T06:13:08.482 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-30T06:13:08.482 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2023-10-30T06:13:08.482 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-30T06:13:08.483 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:08.483 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:08.484 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:08.484 INFO:teuthology.orchestra.run.smithi112.stdout:Install 7 Packages 2023-10-30T06:13:08.484 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:08.485 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 2.1 M 2023-10-30T06:13:08.485 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 8.6 M 2023-10-30T06:13:08.485 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:08.626 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:04 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:08.756 INFO:teuthology.orchestra.run.smithi112.stdout:(1/7): ceph-prometheus-alerts-18.0.0-6858.gd0db 68 kB/s | 15 kB 00:00 2023-10-30T06:13:08.764 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:08.765 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:08.765 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:08.766 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:08.766 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:08.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 1.8 M 2023-10-30T06:13:08.766 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:13:08.767 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 23 k 2023-10-30T06:13:08.767 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 15 k 2023-10-30T06:13:08.767 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-30T06:13:08.767 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-30T06:13:08.767 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-30T06:13:08.768 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2023-10-30T06:13:08.768 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-30T06:13:08.768 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:08.768 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:08.768 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:08.769 INFO:teuthology.orchestra.run.smithi079.stdout:Install 7 Packages 2023-10-30T06:13:08.769 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:08.770 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 2.1 M 2023-10-30T06:13:08.770 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 8.6 M 2023-10-30T06:13:08.770 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:08.832 INFO:teuthology.orchestra.run.smithi112.stdout:(2/7): ceph-grafana-dashboards-18.0.0-6858.gd0d 78 kB/s | 23 kB 00:00 2023-10-30T06:13:08.857 INFO:teuthology.orchestra.run.smithi112.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 90 kB/s | 9.0 kB 00:00 2023-10-30T06:13:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 408 kB/s | 41 kB 00:00 2023-10-30T06:13:08.982 INFO:teuthology.orchestra.run.smithi112.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 245 kB/s | 31 kB 00:00 2023-10-30T06:13:09.082 INFO:teuthology.orchestra.run.smithi112.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2023-10-30T06:13:09.267 INFO:teuthology.orchestra.run.smithi079.stdout:(1/7): ceph-grafana-dashboards-18.0.0-6858.gd0d 68 kB/s | 23 kB 00:00 2023-10-30T06:13:09.376 INFO:teuthology.orchestra.run.smithi079.stdout:(2/7): ceph-prometheus-alerts-18.0.0-6858.gd0db 34 kB/s | 15 kB 00:00 2023-10-30T06:13:09.651 INFO:teuthology.orchestra.run.smithi079.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 24 kB/s | 9.0 kB 00:00 2023-10-30T06:13:09.835 INFO:teuthology.orchestra.run.smithi079.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 168 kB/s | 31 kB 00:00 2023-10-30T06:13:09.868 INFO:teuthology.orchestra.run.smithi079.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 87 kB/s | 41 kB 00:00 2023-10-30T06:13:10.074 INFO:teuthology.orchestra.run.smithi112.stdout:(7/7): ceph-mgr-dashboard-18.0.0-6858.gd0db9f2f 1.1 MB/s | 1.8 MB 00:01 2023-10-30T06:13:10.076 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:10.077 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.3 MB/s | 2.1 MB 00:01 2023-10-30T06:13:10.077 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:10.086 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:10.086 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:10.162 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:10.162 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:10.193 INFO:teuthology.orchestra.run.smithi079.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 526 kB/s | 188 kB 00:00 2023-10-30T06:13:10.235 INFO:teuthology.orchestra.run.smithi079.stdout:(7/7): ceph-mgr-dashboard-18.0.0-6858.gd0db9f2f 1.3 MB/s | 1.8 MB 00:01 2023-10-30T06:13:10.239 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:10.239 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.4 MB/s | 2.1 MB 00:01 2023-10-30T06:13:10.239 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:10.247 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:10.247 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:10.316 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:10.317 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:10.461 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-30T06:13:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-30T06:13:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-30T06:13:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-30T06:13:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.n 5/7 2023-10-30T06:13:10.686 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-10-30T06:13:10.721 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-10-30T06:13:10.776 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-10-30T06:13:10.807 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-10-30T06:13:10.836 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.n 5/7 2023-10-30T06:13:10.913 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9. 6/7 2023-10-30T06:13:10.944 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 7/7 2023-10-30T06:13:11.074 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9. 6/7 2023-10-30T06:13:11.098 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 7/7 2023-10-30T06:13:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 7/7 2023-10-30T06:13:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9. 1/7 2023-10-30T06:13:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 2/7 2023-10-30T06:13:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.n 3/7 2023-10-30T06:13:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-30T06:13:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-30T06:13:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-30T06:13:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 7/7 2023-10-30T06:13:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9. 1/7 2023-10-30T06:13:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarc 2/7 2023-10-30T06:13:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.n 3/7 2023-10-30T06:13:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-10-30T06:13:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-10-30T06:13:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-10-30T06:13:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-30T06:13:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:11.770 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-30T06:13:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-30T06:13:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-30T06:13:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-30T06:13:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:11.772 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:11.932 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-10-30T06:13:11.932 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:11.933 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:11.933 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.933 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.933 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:11.933 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-30T06:13:11.934 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-30T06:13:11.934 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-30T06:13:11.934 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-30T06:13:11.934 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:11.934 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:11.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-30T06:13:12.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-diskprediction-local 2023-10-30T06:13:12.511 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:09 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:12.653 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:12.654 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2023-10-30T06:13:12.654 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:12.654 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2023-10-30T06:13:12.655 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 7.4 M 2023-10-30T06:13:12.655 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-30T06:13:12.655 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-30T06:13:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-30T06:13:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-30T06:13:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-30T06:13:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-30T06:13:12.656 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:12.657 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:12.658 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2023-10-30T06:13:12.658 INFO:teuthology.orchestra.run.smithi112.stdout:Install 12 Packages 2023-10-30T06:13:12.658 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:12.659 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 37 M 2023-10-30T06:13:12.659 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 209 M 2023-10-30T06:13:12.659 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:12.693 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:08 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:12.764 INFO:teuthology.orchestra.run.smithi112.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 295 kB/s | 32 kB 00:00 2023-10-30T06:13:12.814 INFO:teuthology.orchestra.run.smithi112.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 646 kB/s | 17 kB 00:00 2023-10-30T06:13:12.832 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:12.833 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2023-10-30T06:13:12.834 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:12.834 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2023-10-30T06:13:12.834 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:12.834 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 7.4 M 2023-10-30T06:13:12.835 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:13:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-30T06:13:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-30T06:13:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-30T06:13:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-30T06:13:12.836 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-30T06:13:12.837 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-30T06:13:12.837 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-30T06:13:12.837 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-30T06:13:12.838 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-30T06:13:12.838 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-30T06:13:12.838 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-30T06:13:12.838 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:12.838 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:============================================================================================ 2023-10-30T06:13:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:Install 12 Packages 2023-10-30T06:13:12.839 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:12.840 INFO:teuthology.orchestra.run.smithi112.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2023-10-30T06:13:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 37 M 2023-10-30T06:13:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 209 M 2023-10-30T06:13:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:12.915 INFO:teuthology.orchestra.run.smithi112.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2023-10-30T06:13:12.953 INFO:teuthology.orchestra.run.smithi079.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 270 kB/s | 32 kB 00:00 2023-10-30T06:13:13.012 INFO:teuthology.orchestra.run.smithi079.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 483 kB/s | 17 kB 00:00 2023-10-30T06:13:13.014 INFO:teuthology.orchestra.run.smithi112.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.0 MB/s | 205 kB 00:00 2023-10-30T06:13:13.054 INFO:teuthology.orchestra.run.smithi079.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2023-10-30T06:13:13.090 INFO:teuthology.orchestra.run.smithi112.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-10-30T06:13:13.179 INFO:teuthology.orchestra.run.smithi079.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2023-10-30T06:13:13.190 INFO:teuthology.orchestra.run.smithi112.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 4.8 MB 00:00 2023-10-30T06:13:13.216 INFO:teuthology.orchestra.run.smithi112.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 217 kB 00:00 2023-10-30T06:13:13.279 INFO:teuthology.orchestra.run.smithi079.stdout:(5/12): python3-devel-3.9.17-2.el9.x86_64.rpm 2.0 MB/s | 205 kB 00:00 2023-10-30T06:13:13.463 INFO:teuthology.orchestra.run.smithi079.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-10-30T06:13:13.482 INFO:teuthology.orchestra.run.smithi112.stdout:(9/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2023-10-30T06:13:13.613 INFO:teuthology.orchestra.run.smithi079.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2023-10-30T06:13:13.633 INFO:teuthology.orchestra.run.smithi112.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-10-30T06:13:13.639 INFO:teuthology.orchestra.run.smithi079.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.2 MB/s | 217 kB 00:00 2023-10-30T06:13:13.658 INFO:teuthology.orchestra.run.smithi112.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2023-10-30T06:13:13.806 INFO:teuthology.orchestra.run.smithi079.stdout:(9/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2023-10-30T06:13:14.064 INFO:teuthology.orchestra.run.smithi079.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-10-30T06:13:14.098 INFO:teuthology.orchestra.run.smithi079.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 673 kB/s | 195 kB 00:00 2023-10-30T06:13:16.458 INFO:teuthology.orchestra.run.smithi112.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-6 1.9 MB/s | 7.4 MB 00:03 2023-10-30T06:13:16.461 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:16.461 INFO:teuthology.orchestra.run.smithi112.stdout:Total 9.7 MB/s | 37 MB 00:03 2023-10-30T06:13:16.462 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:16.487 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:16.487 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:16.908 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:16.908 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:17.622 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:17.697 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-30T06:13:17.720 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-30T06:13:17.812 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-30T06:13:17.848 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-30T06:13:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-30T06:13:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-30T06:13:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-30T06:13:18.306 INFO:teuthology.orchestra.run.smithi079.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-6 1.4 MB/s | 7.4 MB 00:05 2023-10-30T06:13:18.308 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:18.309 INFO:teuthology.orchestra.run.smithi079.stdout:Total 6.8 MB/s | 37 MB 00:05 2023-10-30T06:13:18.309 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:18.334 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:18.335 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:18.755 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:18.756 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:18.779 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-30T06:13:18.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-30T06:13:19.460 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:19.546 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-10-30T06:13:19.585 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-10-30T06:13:19.679 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-10-30T06:13:19.731 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-10-30T06:13:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-30T06:13:19.946 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-30T06:13:20.032 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-10-30T06:13:20.121 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-30T06:13:20.685 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-10-30T06:13:20.803 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-10-30T06:13:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-30T06:13:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 12/12 2023-10-30T06:13:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-10-30T06:13:23.070 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-10-30T06:13:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 12/12 2023-10-30T06:13:23.666 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 12/12 2023-10-30T06:13:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 1/12 2023-10-30T06:13:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-30T06:13:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-30T06:13:23.667 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-30T06:13:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-30T06:13:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-30T06:13:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-30T06:13:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-30T06:13:23.668 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-30T06:13:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-30T06:13:23.669 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-30T06:13:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-30T06:13:24.125 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:24.126 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-30T06:13:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-30T06:13:24.126 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-30T06:13:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-30T06:13:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-30T06:13:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-30T06:13:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-30T06:13:24.127 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-30T06:13:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-30T06:13:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-30T06:13:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-30T06:13:24.128 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:24.128 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:24.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-rook 2023-10-30T06:13:25.052 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:22 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:25.191 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:25.192 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:25.192 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:25.192 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:25.193 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 48 k 2023-10-30T06:13:25.193 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-30T06:13:25.193 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-30T06:13:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-30T06:13:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-30T06:13:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-30T06:13:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-30T06:13:25.194 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-30T06:13:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-30T06:13:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-30T06:13:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-30T06:13:25.195 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-30T06:13:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-30T06:13:25.196 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:25.196 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:25.196 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:25.196 INFO:teuthology.orchestra.run.smithi112.stdout:Install 13 Packages 2023-10-30T06:13:25.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:25.197 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 2.0 M 2023-10-30T06:13:25.197 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 26 M 2023-10-30T06:13:25.198 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:25.752 INFO:teuthology.orchestra.run.smithi112.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 143 kB/s | 24 kB 00:00 2023-10-30T06:13:25.778 INFO:teuthology.orchestra.run.smithi112.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 92 kB/s | 18 kB 00:00 2023-10-30T06:13:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 12/12 2023-10-30T06:13:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9 1/12 2023-10-30T06:13:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-10-30T06:13:25.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-10-30T06:13:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-10-30T06:13:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-10-30T06:13:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-10-30T06:13:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 7/12 2023-10-30T06:13:25.782 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 8/12 2023-10-30T06:13:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 9/12 2023-10-30T06:13:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 10/12 2023-10-30T06:13:25.783 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 11/12 2023-10-30T06:13:25.820 INFO:teuthology.orchestra.run.smithi112.stdout:(3/13): ceph-mgr-rook-18.0.0-6858.gd0db9f2f.el9 200 kB/s | 48 kB 00:00 2023-10-30T06:13:25.862 INFO:teuthology.orchestra.run.smithi112.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 174 kB 00:00 2023-10-30T06:13:25.887 INFO:teuthology.orchestra.run.smithi112.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2023-10-30T06:13:25.912 INFO:teuthology.orchestra.run.smithi112.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.2 MB/s | 211 kB 00:00 2023-10-30T06:13:25.929 INFO:teuthology.orchestra.run.smithi112.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 638 kB/s | 43 kB 00:00 2023-10-30T06:13:26.171 INFO:teuthology.orchestra.run.smithi112.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 113 kB/s | 32 kB 00:00 2023-10-30T06:13:26.188 INFO:teuthology.orchestra.run.smithi112.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 54 kB/s | 15 kB 00:00 2023-10-30T06:13:26.224 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 12/12 2023-10-30T06:13:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:26.225 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:26.225 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-30T06:13:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-30T06:13:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-30T06:13:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-30T06:13:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-30T06:13:26.226 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-30T06:13:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-30T06:13:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-30T06:13:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-30T06:13:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-30T06:13:26.227 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-30T06:13:26.228 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:26.228 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:26.380 INFO:teuthology.orchestra.run.smithi112.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 308 kB/s | 59 kB 00:00 2023-10-30T06:13:26.405 INFO:teuthology.orchestra.run.smithi112.stdout:(11/13): python3-google-auth-2.23.3-1.el9.noarc 414 kB/s | 197 kB 00:00 2023-10-30T06:13:26.572 INFO:teuthology.orchestra.run.smithi112.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 467 kB/s | 90 kB 00:00 2023-10-30T06:13:26.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-rook 2023-10-30T06:13:26.722 INFO:teuthology.orchestra.run.smithi112.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2023-10-30T06:13:26.724 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:26.725 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2023-10-30T06:13:26.725 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:26.738 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:26.738 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:26.835 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:26.836 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:27.164 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:23 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:27.315 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:27.316 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:27.317 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:27.317 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 48 k 2023-10-30T06:13:27.318 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:13:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-30T06:13:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-30T06:13:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-30T06:13:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-30T06:13:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-30T06:13:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-30T06:13:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-30T06:13:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-30T06:13:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-30T06:13:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-30T06:13:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-30T06:13:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-30T06:13:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:27.321 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:27.321 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:27.321 INFO:teuthology.orchestra.run.smithi079.stdout:Install 13 Packages 2023-10-30T06:13:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:27.323 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 2.0 M 2023-10-30T06:13:27.324 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 26 M 2023-10-30T06:13:27.324 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:27.352 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-30T06:13:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-30T06:13:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-30T06:13:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-30T06:13:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-30T06:13:27.769 INFO:teuthology.orchestra.run.smithi079.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 126 kB/s | 24 kB 00:00 2023-10-30T06:13:27.794 INFO:teuthology.orchestra.run.smithi079.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 83 kB/s | 18 kB 00:00 2023-10-30T06:13:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-30T06:13:27.819 INFO:teuthology.orchestra.run.smithi079.stdout:(3/13): ceph-mgr-rook-18.0.0-6858.gd0db9f2f.el9 195 kB/s | 48 kB 00:00 2023-10-30T06:13:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 7/13 2023-10-30T06:13:27.919 INFO:teuthology.orchestra.run.smithi079.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 174 kB 00:00 2023-10-30T06:13:27.945 INFO:teuthology.orchestra.run.smithi079.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 877 kB/s | 132 kB 00:00 2023-10-30T06:13:27.970 INFO:teuthology.orchestra.run.smithi079.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 211 kB 00:00 2023-10-30T06:13:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-30T06:13:27.996 INFO:teuthology.orchestra.run.smithi079.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 565 kB/s | 43 kB 00:00 2023-10-30T06:13:28.254 INFO:teuthology.orchestra.run.smithi079.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 53 kB/s | 15 kB 00:00 2023-10-30T06:13:28.288 INFO:teuthology.orchestra.run.smithi079.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 94 kB/s | 32 kB 00:00 2023-10-30T06:13:28.539 INFO:teuthology.orchestra.run.smithi079.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 236 kB/s | 59 kB 00:00 2023-10-30T06:13:28.572 INFO:teuthology.orchestra.run.smithi079.stdout:(11/13): python3-google-auth-2.23.3-1.el9.noarc 342 kB/s | 197 kB 00:00 2023-10-30T06:13:28.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-30T06:13:28.747 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-30T06:13:28.773 INFO:teuthology.orchestra.run.smithi079.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 384 kB/s | 90 kB 00:00 2023-10-30T06:13:28.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-30T06:13:28.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-30T06:13:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 13/13 2023-10-30T06:13:29.031 INFO:teuthology.orchestra.run.smithi079.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.3 MB/s | 1.0 MB 00:00 2023-10-30T06:13:29.033 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:29.035 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.2 MB/s | 2.0 MB 00:01 2023-10-30T06:13:29.035 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:29.048 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:29.048 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:29.146 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:29.147 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 13/13 2023-10-30T06:13:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/13 2023-10-30T06:13:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-30T06:13:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-30T06:13:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-30T06:13:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-30T06:13:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-30T06:13:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-30T06:13:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-30T06:13:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-30T06:13:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 10/13 2023-10-30T06:13:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-30T06:13:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-30T06:13:29.677 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-10-30T06:13:29.916 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-10-30T06:13:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-30T06:13:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:29.954 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-30T06:13:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-30T06:13:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-30T06:13:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-30T06:13:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-30T06:13:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-30T06:13:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-30T06:13:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:29.957 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:29.971 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-10-30T06:13:30.017 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-10-30T06:13:30.058 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-10-30T06:13:30.151 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-10-30T06:13:30.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-cephadm 2023-10-30T06:13:30.253 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 7/13 2023-10-30T06:13:30.302 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-10-30T06:13:30.733 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:27 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:30.874 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:30.875 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:30.875 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:30.875 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:30.876 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:30.876 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 132 k 2023-10-30T06:13:30.876 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:30.876 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-30T06:13:30.876 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-30T06:13:30.877 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-30T06:13:30.877 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-30T06:13:30.877 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-30T06:13:30.877 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:30.877 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:30.878 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:13:30.878 INFO:teuthology.orchestra.run.smithi112.stdout:Install 6 Packages 2023-10-30T06:13:30.878 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:30.878 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 6.8 M 2023-10-30T06:13:30.878 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 33 M 2023-10-30T06:13:30.879 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:31.021 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-10-30T06:13:31.075 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-10-30T06:13:31.106 INFO:teuthology.orchestra.run.smithi112.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.3 MB/s | 228 kB 00:00 2023-10-30T06:13:31.109 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-10-30T06:13:31.154 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-10-30T06:13:31.179 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 13/13 2023-10-30T06:13:31.198 INFO:teuthology.orchestra.run.smithi112.stdout:(2/6): ceph-mgr-cephadm-18.0.0-6858.gd0db9f2f.e 490 kB/s | 132 kB 00:00 2023-10-30T06:13:31.315 INFO:teuthology.orchestra.run.smithi112.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2023-10-30T06:13:31.340 INFO:teuthology.orchestra.run.smithi112.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 412 kB/s | 58 kB 00:00 2023-10-30T06:13:31.407 INFO:teuthology.orchestra.run.smithi112.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2023-10-30T06:13:31.432 INFO:teuthology.orchestra.run.smithi112.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 436 kB/s | 51 kB 00:00 2023-10-30T06:13:31.434 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:31.435 INFO:teuthology.orchestra.run.smithi112.stdout:Total 12 MB/s | 6.8 MB 00:00 2023-10-30T06:13:31.435 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:31.444 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:31.444 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:31.532 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:31.532 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:31.828 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:31.988 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-30T06:13:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 13/13 2023-10-30T06:13:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/13 2023-10-30T06:13:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-10-30T06:13:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-10-30T06:13:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-10-30T06:13:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-10-30T06:13:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-10-30T06:13:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-10-30T06:13:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-10-30T06:13:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-10-30T06:13:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 10/13 2023-10-30T06:13:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-10-30T06:13:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-10-30T06:13:32.059 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-30T06:13:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-10-30T06:13:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:32.373 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:32.373 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-30T06:13:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-30T06:13:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-30T06:13:32.374 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-30T06:13:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-30T06:13:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-30T06:13:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-30T06:13:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-30T06:13:32.375 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-30T06:13:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-30T06:13:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-30T06:13:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-30T06:13:32.376 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:32.377 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:32.570 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-30T06:13:32.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-cephadm 2023-10-30T06:13:32.661 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-30T06:13:32.723 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-30T06:13:32.746 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 6/6 2023-10-30T06:13:33.169 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:29 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:33.309 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:33.309 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:33.310 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:33.310 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:33.310 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:33.310 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 132 k 2023-10-30T06:13:33.310 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:13:33.311 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-30T06:13:33.311 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-30T06:13:33.311 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-30T06:13:33.311 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-30T06:13:33.311 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-30T06:13:33.312 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:33.312 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:33.312 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:13:33.312 INFO:teuthology.orchestra.run.smithi079.stdout:Install 6 Packages 2023-10-30T06:13:33.312 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:33.313 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 6.8 M 2023-10-30T06:13:33.313 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 33 M 2023-10-30T06:13:33.313 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:33.496 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 6/6 2023-10-30T06:13:33.496 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/6 2023-10-30T06:13:33.496 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-30T06:13:33.497 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-30T06:13:33.497 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-30T06:13:33.498 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-30T06:13:33.697 INFO:teuthology.orchestra.run.smithi079.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.2 MB/s | 228 kB 00:00 2023-10-30T06:13:33.792 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-30T06:13:33.793 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:33.793 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:33.793 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:33.793 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-30T06:13:33.794 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-30T06:13:33.794 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-30T06:13:33.794 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-30T06:13:33.794 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-30T06:13:33.794 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:33.795 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:33.911 INFO:teuthology.orchestra.run.smithi079.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2023-10-30T06:13:33.978 INFO:teuthology.orchestra.run.smithi079.stdout:(3/6): ceph-mgr-cephadm-18.0.0-6858.gd0db9f2f.e 282 kB/s | 132 kB 00:00 2023-10-30T06:13:34.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-fuse 2023-10-30T06:13:34.254 INFO:teuthology.orchestra.run.smithi079.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 169 kB/s | 58 kB 00:00 2023-10-30T06:13:34.287 INFO:teuthology.orchestra.run.smithi079.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 926 kB/s | 548 kB 00:00 2023-10-30T06:13:34.312 INFO:teuthology.orchestra.run.smithi079.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 153 kB/s | 51 kB 00:00 2023-10-30T06:13:34.314 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:34.314 INFO:teuthology.orchestra.run.smithi079.stdout:Total 6.7 MB/s | 6.8 MB 00:01 2023-10-30T06:13:34.315 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:34.324 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:34.324 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:34.413 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:34.415 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:34.559 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:31 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:34.697 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:34.698 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:34.699 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:34.699 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:34.699 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:34.699 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 856 k 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:34.700 INFO:teuthology.orchestra.run.smithi112.stdout:Install 2 Packages 2023-10-30T06:13:34.701 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:34.701 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 934 k 2023-10-30T06:13:34.701 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.7 M 2023-10-30T06:13:34.701 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:34.707 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:34.826 INFO:teuthology.orchestra.run.smithi112.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 625 kB/s | 79 kB 00:00 2023-10-30T06:13:34.874 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-10-30T06:13:34.930 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-10-30T06:13:35.193 INFO:teuthology.orchestra.run.smithi112.stdout:(2/2): ceph-fuse-18.0.0-6858.gd0db9f2f.el9.x86_ 1.7 MB/s | 856 kB 00:00 2023-10-30T06:13:35.194 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:35.194 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.8 MB/s | 934 kB 00:00 2023-10-30T06:13:35.194 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:35.214 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:35.214 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:35.290 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:35.290 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:35.433 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-10-30T06:13:35.469 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:35.507 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-10-30T06:13:35.568 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-10-30T06:13:35.594 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 6/6 2023-10-30T06:13:35.657 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-30T06:13:35.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:13:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:13:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/2 2023-10-30T06:13:36.421 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 6/6 2023-10-30T06:13:36.421 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/6 2023-10-30T06:13:36.422 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-10-30T06:13:36.422 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-10-30T06:13:36.422 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-10-30T06:13:36.422 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-10-30T06:13:36.666 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-30T06:13:36.667 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:36.667 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:36.667 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-30T06:13:36.667 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:36.667 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:36.726 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-10-30T06:13:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:36.727 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-30T06:13:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-30T06:13:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-30T06:13:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-30T06:13:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-30T06:13:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:36.729 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:36.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-volume 2023-10-30T06:13:37.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-fuse 2023-10-30T06:13:37.474 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:34 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:37.569 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:33 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:37.619 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:37.620 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:37.620 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-10-30T06:13:37.621 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:37.621 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:37.621 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 262 k 2023-10-30T06:13:37.621 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:37.622 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:37.622 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:37.622 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:37.622 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:37.622 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 262 k 2023-10-30T06:13:37.623 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 1.2 M 2023-10-30T06:13:37.623 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:37.711 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:37.712 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:37.713 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:37.713 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 856 k 2023-10-30T06:13:37.713 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:13:37.713 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-30T06:13:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:37.714 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:37.714 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:37.714 INFO:teuthology.orchestra.run.smithi079.stdout:Install 2 Packages 2023-10-30T06:13:37.714 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:37.715 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 934 k 2023-10-30T06:13:37.715 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.7 M 2023-10-30T06:13:37.715 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:37.848 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-volume-18.0.0-6858.gd0db9f2f.el9.noarch.rp 1.1 MB/s | 262 kB 00:00 2023-10-30T06:13:37.849 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:37.849 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.1 MB/s | 262 kB 00:00 2023-10-30T06:13:37.849 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:37.857 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:37.857 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:37.895 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:37.895 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:37.924 INFO:teuthology.orchestra.run.smithi079.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 372 kB/s | 79 kB 00:00 2023-10-30T06:13:38.113 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:38.175 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:38.190 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:38.191 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:38.191 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-30T06:13:38.191 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:39.176 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:39.176 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:39.341 INFO:teuthology.orchestra.run.smithi079.stdout:(2/2): ceph-fuse-18.0.0-6858.gd0db9f2f.el9.x86_ 526 kB/s | 856 kB 00:01 2023-10-30T06:13:39.343 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:39.344 INFO:teuthology.orchestra.run.smithi079.stdout:Total 573 kB/s | 934 kB 00:01 2023-10-30T06:13:39.344 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:39.368 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:39.368 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:39.446 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:39.447 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:39.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados-devel 2023-10-30T06:13:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:39.765 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-10-30T06:13:39.841 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:13:40.038 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:37 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:40.178 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:40.179 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:40.180 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:13:40.180 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:40.180 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:40.180 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 126 k 2023-10-30T06:13:40.180 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:40.181 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:40.181 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:40.181 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:40.181 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:40.181 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 126 k 2023-10-30T06:13:40.182 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 456 k 2023-10-30T06:13:40.182 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:40.402 INFO:teuthology.orchestra.run.smithi112.stdout:librados-devel-18.0.0-6858.gd0db9f2f.el9.x86_64 571 kB/s | 126 kB 00:00 2023-10-30T06:13:40.403 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:40.403 INFO:teuthology.orchestra.run.smithi112.stdout:Total 566 kB/s | 126 kB 00:00 2023-10-30T06:13:40.404 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:40.409 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:40.409 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:13:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/2 2023-10-30T06:13:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:40.726 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:40.842 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-10-30T06:13:40.843 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:40.843 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:40.843 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-10-30T06:13:40.843 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:40.844 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:41.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-volume 2023-10-30T06:13:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:41.447 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:41.448 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:41.448 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:41.448 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:41.448 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:41.448 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:41.678 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:37 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:41.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install libcephfs2 2023-10-30T06:13:41.821 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:41.822 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-10-30T06:13:41.823 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:41.823 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:18.0.0-6858.gd0db9f2f.el9 ceph-noarch 262 k 2023-10-30T06:13:41.823 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:41.824 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:41.824 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:41.824 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:41.824 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:41.824 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 262 k 2023-10-30T06:13:41.825 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.2 M 2023-10-30T06:13:41.825 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:42.056 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-volume-18.0.0-6858.gd0db9f2f.el9.noarch.rp 1.1 MB/s | 262 kB 00:00 2023-10-30T06:13:42.057 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:42.058 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.1 MB/s | 262 kB 00:00 2023-10-30T06:13:42.058 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:42.065 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:42.065 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:42.099 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:42.099 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:42.252 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:39 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:42.309 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:42.344 INFO:teuthology.orchestra.run.smithi112.stdout:Package libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:42.375 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:42.391 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:42.392 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:42.392 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-30T06:13:42.392 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:42.393 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:42.394 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:42.394 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:42.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install libcephfs-devel 2023-10-30T06:13:43.001 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:39 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:43.141 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:43.142 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:43.142 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:13:43.142 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:43.143 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:43.143 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 31 k 2023-10-30T06:13:43.143 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:43.143 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:43.143 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:43.144 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:43.144 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:43.144 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 31 k 2023-10-30T06:13:43.144 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 139 k 2023-10-30T06:13:43.145 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:43.314 INFO:teuthology.orchestra.run.smithi112.stdout:libcephfs-devel-18.0.0-6858.gd0db9f2f.el9.x86_6 182 kB/s | 31 kB 00:00 2023-10-30T06:13:43.314 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:43.315 INFO:teuthology.orchestra.run.smithi112.stdout:Total 181 kB/s | 31 kB 00:00 2023-10-30T06:13:43.315 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:43.318 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:43.318 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:43.356 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:43.357 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:43.442 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 1/1 2023-10-30T06:13:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:43.443 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:18.0.0-6858.gd0db9f2f.el9.noarch 2023-10-30T06:13:43.443 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:43.443 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:43.570 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:43.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados-devel 2023-10-30T06:13:43.934 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:44.199 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:44.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:44.200 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:44.200 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:44.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:44.200 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:44.312 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:40 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:44.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados2 2023-10-30T06:13:44.450 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:44.451 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:44.451 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 126 k 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:44.452 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:44.453 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:44.453 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:44.453 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 126 k 2023-10-30T06:13:44.453 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 456 k 2023-10-30T06:13:44.453 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:44.815 INFO:teuthology.orchestra.run.smithi079.stdout:librados-devel-18.0.0-6858.gd0db9f2f.el9.x86_64 349 kB/s | 126 kB 00:00 2023-10-30T06:13:44.815 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:44.816 INFO:teuthology.orchestra.run.smithi079.stdout:Total 347 kB/s | 126 kB 00:00 2023-10-30T06:13:44.816 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:44.821 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:44.821 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:44.876 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:44.876 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:44.908 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:41 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:44.999 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:45.027 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:45.049 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:45.050 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:45.050 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:45.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librbd1 2023-10-30T06:13:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:45.650 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:42 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:45.740 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:45.789 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:45.790 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:45.791 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:45.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rados 2023-10-30T06:13:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:45.878 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:45.879 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:45.879 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:45.879 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:46.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs2 2023-10-30T06:13:46.387 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:43 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:46.477 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:46.526 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:46.527 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:46.527 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:46.581 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rgw 2023-10-30T06:13:46.689 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:42 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:46.780 INFO:teuthology.orchestra.run.smithi079.stdout:Package libcephfs2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:46.833 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:46.834 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:46.834 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:46.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs-devel 2023-10-30T06:13:47.116 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:44 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:47.205 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:47.256 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:47.256 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:47.257 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:47.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-cephfs 2023-10-30T06:13:47.456 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:43 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:47.594 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:47.595 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:47.595 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:13:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:47.596 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 31 k 2023-10-30T06:13:47.596 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:47.596 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 31 k 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 139 k 2023-10-30T06:13:47.597 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:47.770 INFO:teuthology.orchestra.run.smithi079.stdout:libcephfs-devel-18.0.0-6858.gd0db9f2f.el9.x86_6 179 kB/s | 31 kB 00:00 2023-10-30T06:13:47.771 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:47.771 INFO:teuthology.orchestra.run.smithi079.stdout:Total 177 kB/s | 31 kB 00:00 2023-10-30T06:13:47.771 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:47.774 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:47.775 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:47.808 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:47.809 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:47.854 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:44 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:47.896 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:47.944 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:47.993 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:47.993 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:47.994 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:48.026 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:48.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rbd 2023-10-30T06:13:48.443 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:48.584 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:45 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:48.672 INFO:teuthology.orchestra.run.smithi112.stdout:Package python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:48.716 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:48.717 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:48.717 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:48.717 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:48.717 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:48.717 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:48.723 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:48.724 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:13:48.724 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:48.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-fuse 2023-10-30T06:13:48.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados2 2023-10-30T06:13:49.321 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:46 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:49.461 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:49.462 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:49.462 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2023-10-30T06:13:49.463 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:49.463 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:49.463 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 85 k 2023-10-30T06:13:49.463 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:49.463 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:49.464 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:49.464 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:49.464 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:49.464 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 85 k 2023-10-30T06:13:49.464 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 227 k 2023-10-30T06:13:49.465 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:49.477 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:45 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:49.568 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:49.617 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:49.618 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:49.618 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:49.649 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-fuse-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 454 kB/s | 85 kB 00:00 2023-10-30T06:13:49.650 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:49.651 INFO:teuthology.orchestra.run.smithi112.stdout:Total 451 kB/s | 85 kB 00:00 2023-10-30T06:13:49.651 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:49.657 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:49.657 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:49.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librbd1 2023-10-30T06:13:49.711 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:49.711 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:49.883 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:49.989 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:50.242 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:46 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:50.334 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:50.370 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:50.384 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:50.385 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:50.386 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:50.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rados 2023-10-30T06:13:50.675 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:50.675 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:50.675 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:50.676 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:50.676 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:50.676 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:50.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-mirror 2023-10-30T06:13:51.013 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:47 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:51.102 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rados-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:51.151 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:51.152 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:51.152 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:51.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rgw 2023-10-30T06:13:51.463 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:48 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:51.604 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:51.605 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:51.605 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:13:51.605 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:51.606 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.0 M 2023-10-30T06:13:51.606 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:51.606 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:51.606 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:51.607 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:51.607 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:51.607 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 3.0 M 2023-10-30T06:13:51.607 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 12 M 2023-10-30T06:13:51.607 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:51.775 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:47 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:51.864 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rgw-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:51.913 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:51.914 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:51.914 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:51.968 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-mirror-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 8.3 MB/s | 3.0 MB 00:00 2023-10-30T06:13:51.969 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:51.969 INFO:teuthology.orchestra.run.smithi112.stdout:Total 8.3 MB/s | 3.0 MB 00:00 2023-10-30T06:13:51.970 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:51.978 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:51.978 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:51.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-cephfs 2023-10-30T06:13:52.051 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:52.051 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:52.459 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:52.533 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:48 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:52.543 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:52.543 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-30T06:13:52.543 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-30T06:13:52.543 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-30T06:13:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:52.621 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-cephfs-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:52.670 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:52.671 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:52.671 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:52.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rbd 2023-10-30T06:13:53.292 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:49 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:53.381 INFO:teuthology.orchestra.run.smithi079.stdout:Package python3-rbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:13:53.431 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:53.432 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:13:53.432 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:53.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-fuse 2023-10-30T06:13:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:53.689 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:53.690 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:53.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-nbd 2023-10-30T06:13:54.063 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:50 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:54.206 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:54.208 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:54.208 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2023-10-30T06:13:54.208 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:54.208 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:54.208 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 85 k 2023-10-30T06:13:54.209 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:54.209 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:54.209 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:54.209 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:54.210 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:54.210 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 85 k 2023-10-30T06:13:54.210 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 227 k 2023-10-30T06:13:54.210 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:54.474 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-fuse-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 319 kB/s | 85 kB 00:00 2023-10-30T06:13:54.474 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:54.475 INFO:teuthology.orchestra.run.smithi079.stdout:Total 317 kB/s | 85 kB 00:00 2023-10-30T06:13:54.475 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:54.481 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:54.512 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:51 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:54.536 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:54.536 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:54.654 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:54.655 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:54.655 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2023-10-30T06:13:54.655 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:54.655 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:54.655 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 169 k 2023-10-30T06:13:54.656 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:54.656 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:54.656 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:54.656 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:13:54.656 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:54.657 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 169 k 2023-10-30T06:13:54.657 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 486 k 2023-10-30T06:13:54.657 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:54.671 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:55.188 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-nbd-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 316 kB/s | 169 kB 00:00 2023-10-30T06:13:55.189 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:55.189 INFO:teuthology.orchestra.run.smithi112.stdout:Total 316 kB/s | 169 kB 00:00 2023-10-30T06:13:55.189 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:13:55.197 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:13:55.197 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:13:55.209 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:55.252 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:13:55.252 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:13:55.408 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:13:55.519 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:55.533 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:55.533 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:55.534 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:13:55.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-mirror 2023-10-30T06:13:55.945 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:56.241 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:56.242 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:56.242 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:13:56.242 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:56.242 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:56.243 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:13:56.353 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:52 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:13:56.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-base-debuginfo 2023-10-30T06:13:56.501 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:13:56.502 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:56.502 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:13:56.503 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:56.503 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:13:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 3.0 M 2023-10-30T06:13:56.503 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:56.503 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:13:56.504 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:13:56.504 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:13:56.504 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:56.504 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 3.0 M 2023-10-30T06:13:56.504 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 12 M 2023-10-30T06:13:56.505 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:13:56.994 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:01:53 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:13:57.137 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:13:57.137 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:13:57.138 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:57.138 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:13:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 74 M 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 20 M 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:13:57.139 INFO:teuthology.orchestra.run.smithi112.stdout:Install 2 Packages 2023-10-30T06:13:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:13:57.140 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 94 M 2023-10-30T06:13:57.140 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 511 M 2023-10-30T06:13:57.140 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:13:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-mirror-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 1.8 MB/s | 3.0 MB 00:01 2023-10-30T06:13:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:13:58.166 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.8 MB/s | 3.0 MB 00:01 2023-10-30T06:13:58.167 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:13:58.173 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:13:58.173 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:13:58.245 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:13:58.246 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:13:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:13:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:58.755 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-30T06:13:58.756 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-30T06:13:58.756 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-30T06:13:58.756 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-30T06:13:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:59.919 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:13:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:59.920 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:13:59.920 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:13:59.921 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:13:59.921 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:14:00.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-nbd 2023-10-30T06:14:00.753 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:56 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:14:00.893 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:14:00.894 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:00.894 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2023-10-30T06:14:00.894 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:00.894 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:14:00.895 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 169 k 2023-10-30T06:14:00.895 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:00.895 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:14:00.895 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:00.896 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:14:00.896 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:00.896 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 169 k 2023-10-30T06:14:00.896 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 486 k 2023-10-30T06:14:00.896 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:14:01.926 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-nbd-18.0.0-6858.gd0db9f2f.el9.x86_64.rpm 163 kB/s | 169 kB 00:01 2023-10-30T06:14:01.927 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:14:01.927 INFO:teuthology.orchestra.run.smithi079.stdout:Total 163 kB/s | 169 kB 00:01 2023-10-30T06:14:01.927 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:14:01.936 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:14:01.936 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:14:01.996 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:14:01.996 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:14:02.148 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:14:02.258 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:14:02.740 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:14:03.047 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:14:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:03.048 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:14:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:14:03.048 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:03.048 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:14:03.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-base-debuginfo 2023-10-30T06:14:03.374 INFO:teuthology.orchestra.run.smithi112.stdout:(1/2): ceph-debuginfo-18.0.0-6858.gd0db9f2f.el9 3.2 MB/s | 20 MB 00:06 2023-10-30T06:14:03.852 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:01:59 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:14:03.998 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:14:03.999 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:14:03.999 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:03.999 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:14:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 74 M 2023-10-30T06:14:04.000 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-10-30T06:14:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 20 M 2023-10-30T06:14:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:04.000 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:14:04.001 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:04.001 INFO:teuthology.orchestra.run.smithi079.stdout:Install 2 Packages 2023-10-30T06:14:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:04.001 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 94 M 2023-10-30T06:14:04.002 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 511 M 2023-10-30T06:14:04.002 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:14:07.589 INFO:teuthology.orchestra.run.smithi079.stdout:(1/2): ceph-debuginfo-18.0.0-6858.gd0db9f2f.el9 5.6 MB/s | 20 MB 00:03 2023-10-30T06:14:13.556 INFO:teuthology.orchestra.run.smithi112.stdout:(2/2): ceph-base-debuginfo-18.0.0-6858.gd0db9f2 4.5 MB/s | 74 MB 00:16 2023-10-30T06:14:13.558 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:14:13.558 INFO:teuthology.orchestra.run.smithi112.stdout:Total 5.7 MB/s | 94 MB 00:16 2023-10-30T06:14:13.559 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:14:13.563 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:14:13.563 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:14:13.922 INFO:teuthology.orchestra.run.smithi079.stdout:(2/2): ceph-base-debuginfo-18.0.0-6858.gd0db9f2 7.5 MB/s | 74 MB 00:09 2023-10-30T06:14:13.924 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:14:13.924 INFO:teuthology.orchestra.run.smithi079.stdout:Total 9.5 MB/s | 94 MB 00:09 2023-10-30T06:14:13.925 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:14:13.929 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:14:13.929 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:14:14.180 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:14:14.180 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:14:14.544 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:14:14.544 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:14:17.090 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:14:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:14:24.761 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/2 2023-10-30T06:14:24.866 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 2/2 2023-10-30T06:14:25.076 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/2 2023-10-30T06:14:25.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 2/2 2023-10-30T06:14:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 2/2 2023-10-30T06:14:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/2 2023-10-30T06:14:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:14:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:14:28.545 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:14:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:14:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:14:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:14:28.546 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:14:28.744 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 2/2 2023-10-30T06:14:28.744 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/2 2023-10-30T06:14:28.746 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-common-debuginfo 2023-10-30T06:14:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2/2 2023-10-30T06:14:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:29.016 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:14:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:14:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:14:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:29.016 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:14:29.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-common-debuginfo 2023-10-30T06:14:29.307 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:02:26 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:14:29.455 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:14:29.456 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:14:29.457 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:14:29.457 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:14:29.457 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:14:29.457 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 411 M 2023-10-30T06:14:29.457 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:14:29.458 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:14:29.458 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:14:29.458 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:14:29.458 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:14:29.459 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 411 M 2023-10-30T06:14:29.459 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 1.7 G 2023-10-30T06:14:29.459 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:14:29.841 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:02:25 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:14:29.990 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:14:29.991 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:29.992 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:14:29.992 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:29.992 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:14:29.993 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 411 M 2023-10-30T06:14:29.993 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:29.993 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:14:29.994 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:14:29.994 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:14:29.994 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:14:29.995 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 411 M 2023-10-30T06:14:29.995 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.7 G 2023-10-30T06:14:29.995 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:15:02.026 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-common-debuginfo-18.0.0-6858.gd0db9f2f.el9 13 MB/s | 411 MB 00:32 2023-10-30T06:15:02.027 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:15:02.027 INFO:teuthology.orchestra.run.smithi112.stdout:Total 13 MB/s | 411 MB 00:32 2023-10-30T06:15:02.027 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:15:02.032 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:15:02.032 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:15:04.485 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:15:04.485 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:15:15.152 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-common-debuginfo-18.0.0-6858.gd0db9f2f.el9 9.1 MB/s | 411 MB 00:45 2023-10-30T06:15:15.153 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:15:15.154 INFO:teuthology.orchestra.run.smithi079.stdout:Total 9.1 MB/s | 411 MB 00:45 2023-10-30T06:15:15.154 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:15:15.158 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:15:15.159 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:15:17.607 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:15:17.608 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:15:45.751 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:15:45.845 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:15:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:15:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:15:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:53.909 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:15:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:15:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:53.910 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:15:54.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-10-30T06:15:54.697 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:03:51 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:15:54.839 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:15:54.840 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:15:54.840 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:15:54.840 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:15:54.840 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:15:54.840 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.4 M 2023-10-30T06:15:54.841 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:54.841 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:15:54.841 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================== 2023-10-30T06:15:54.841 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:15:54.841 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:54.842 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 1.4 M 2023-10-30T06:15:54.842 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 6.3 M 2023-10-30T06:15:54.842 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:15:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-immutable-object-cache-debuginfo-18.0.0-68 2.0 MB/s | 1.4 MB 00:00 2023-10-30T06:15:55.559 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:15:55.559 INFO:teuthology.orchestra.run.smithi112.stdout:Total 2.0 MB/s | 1.4 MB 00:00 2023-10-30T06:15:55.560 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:15:55.563 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:15:55.563 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:15:55.605 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:15:55.605 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:15:55.862 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:15:55.983 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:15:56.383 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:15:56.632 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:15:56.633 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:56.633 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:15:56.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:15:56.633 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:56.634 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:15:56.817 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw-debuginfo 2023-10-30T06:15:57.361 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:03:54 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:15:57.501 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:15:57.502 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:15:57.502 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:15:57.502 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:15:57.503 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:15:57.503 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 175 M 2023-10-30T06:15:57.503 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:57.503 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:15:57.503 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:15:57.504 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:15:57.504 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:15:57.504 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 175 M 2023-10-30T06:15:57.504 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 734 M 2023-10-30T06:15:57.504 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:15:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:15:59.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:16:06.568 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-radosgw-debuginfo-18.0.0-6858.gd0db9f2f.el 19 MB/s | 175 MB 00:09 2023-10-30T06:16:06.569 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:16:06.570 INFO:teuthology.orchestra.run.smithi112.stdout:Total 19 MB/s | 175 MB 00:09 2023-10-30T06:16:06.570 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:16:06.574 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:16:06.574 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:16:07.642 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:16:07.642 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:16:08.121 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:16:08.387 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:16:08.387 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:08.388 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:16:08.388 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:16:08.388 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:08.388 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:16:08.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-10-30T06:16:09.196 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:04:05 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:16:09.337 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:16:09.338 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:16:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:16:09.338 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:16:09.339 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:16:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.4 M 2023-10-30T06:16:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:09.339 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:16:09.339 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================== 2023-10-30T06:16:09.340 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:16:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:09.340 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 1.4 M 2023-10-30T06:16:09.340 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 6.3 M 2023-10-30T06:16:09.340 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:16:09.657 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-immutable-object-cache-debuginfo-18.0.0-68 4.5 MB/s | 1.4 MB 00:00 2023-10-30T06:16:09.658 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:16:09.658 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.5 MB/s | 1.4 MB 00:00 2023-10-30T06:16:09.658 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:16:09.661 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:16:09.661 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:16:09.702 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:16:09.703 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:16:09.953 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:16:10.089 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:16:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:16:10.784 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.0.0-6858. 1/1 2023-10-30T06:16:10.784 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:10.785 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:16:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:16:10.785 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:10.785 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:16:10.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw-debuginfo 2023-10-30T06:16:11.534 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:04:07 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:16:11.673 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:16:11.674 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:16:11.675 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:11.675 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:16:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 175 M 2023-10-30T06:16:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:11.676 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:16:11.676 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:11.676 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:16:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:11.676 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 175 M 2023-10-30T06:16:11.677 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 734 M 2023-10-30T06:16:11.677 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:16:22.026 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-radosgw-debuginfo-18.0.0-6858.gd0db9f2f.el 17 MB/s | 175 MB 00:10 2023-10-30T06:16:22.027 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:16:22.027 INFO:teuthology.orchestra.run.smithi079.stdout:Total 17 MB/s | 175 MB 00:10 2023-10-30T06:16:22.028 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:16:22.032 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:16:22.032 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:16:23.090 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:16:23.091 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:16:25.535 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:16:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:30.233 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:30.494 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:30.495 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:16:30.495 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:16:30.495 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:16:30.495 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:16:30.495 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:16:30.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-test-debuginfo 2023-10-30T06:16:31.235 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:04:28 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:16:31.375 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:16:31.377 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:16:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:16:31.377 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:16:31.378 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:16:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 627 M 2023-10-30T06:16:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:16:31.379 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:16:31.380 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:16:31.380 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:16:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:16:31.381 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 627 M 2023-10-30T06:16:31.381 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.8 G 2023-10-30T06:16:31.382 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:16:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:16:41.255 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:46.023 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:46.294 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x 1/1 2023-10-30T06:16:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:46.295 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:16:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:16:46.295 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:46.295 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:16:46.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-test-debuginfo 2023-10-30T06:16:47.065 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:04:43 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:16:47.206 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:16:47.207 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:16:47.207 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:47.207 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:16:47.207 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 627 M 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:16:47.208 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 627 M 2023-10-30T06:16:47.209 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.8 G 2023-10-30T06:16:47.209 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:17:03.300 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-test-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 20 MB/s | 627 MB 00:31 2023-10-30T06:17:03.301 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:17:03.301 INFO:teuthology.orchestra.run.smithi112.stdout:Total 20 MB/s | 627 MB 00:31 2023-10-30T06:17:03.302 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:17:03.306 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:17:03.306 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:17:07.022 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:17:07.023 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:17:37.692 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-test-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 12 MB/s | 627 MB 00:50 2023-10-30T06:17:37.693 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:17:37.693 INFO:teuthology.orchestra.run.smithi079.stdout:Total 12 MB/s | 627 MB 00:50 2023-10-30T06:17:37.694 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:17:37.699 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:17:37.699 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:17:41.433 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:17:41.433 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:18:12.632 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:18:12.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:20.304 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:20.557 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:18:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:18:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:20.557 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:18:20.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-base-debuginfo 2023-10-30T06:18:21.345 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:06:18 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:18:21.436 INFO:teuthology.orchestra.run.smithi112.stdout:Package ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:18:21.485 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:18:21.486 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2023-10-30T06:18:21.486 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:18:21.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mgr-debuginfo 2023-10-30T06:18:22.088 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:06:19 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:18:22.228 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:18:22.229 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:18:22.230 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:22.230 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:18:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 19 M 2023-10-30T06:18:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:22.231 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:18:22.231 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:22.231 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:18:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:22.231 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 19 M 2023-10-30T06:18:22.232 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 101 M 2023-10-30T06:18:22.232 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:18:23.496 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-mgr-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 15 MB/s | 19 MB 00:01 2023-10-30T06:18:23.496 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:18:23.497 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 19 MB 00:01 2023-10-30T06:18:23.497 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:18:23.500 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:18:23.501 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:18:23.648 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:18:23.649 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:18:25.833 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:18:25.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:26.946 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:27.196 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:27.196 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:27.197 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:18:27.197 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:18:27.197 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:27.197 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:18:27.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mds-debuginfo 2023-10-30T06:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:06:24 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 47 M 2023-10-30T06:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 47 M 2023-10-30T06:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 170 M 2023-10-30T06:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:18:30.940 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-mds-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 16 MB/s | 47 MB 00:02 2023-10-30T06:18:30.941 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:18:30.941 INFO:teuthology.orchestra.run.smithi112.stdout:Total 16 MB/s | 47 MB 00:02 2023-10-30T06:18:30.942 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:18:30.945 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:18:30.946 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:18:31.257 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:18:31.257 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:18:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:18:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:37.352 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:37.603 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:37.604 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:37.604 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:18:37.604 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:18:37.604 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:37.605 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:18:37.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-mon-debuginfo 2023-10-30T06:18:38.296 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:06:35 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:18:38.436 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:18:38.437 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:38.437 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:18:38.437 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:38.438 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:18:38.438 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 112 M 2023-10-30T06:18:38.438 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:38.438 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:18:38.438 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:18:38.439 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:18:38.439 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:18:38.439 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 112 M 2023-10-30T06:18:38.439 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 509 M 2023-10-30T06:18:38.439 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:18:45.017 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-mon-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 17 MB/s | 112 MB 00:06 2023-10-30T06:18:45.017 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:18:45.018 INFO:teuthology.orchestra.run.smithi112.stdout:Total 17 MB/s | 112 MB 00:06 2023-10-30T06:18:45.018 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:18:45.022 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:18:45.022 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:18:45.718 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:18:45.719 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:18:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:18:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:18:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:18:56.082 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:18:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:18:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:18:56.083 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:18:56.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-base-debuginfo 2023-10-30T06:18:56.902 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:06:52 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:18:56.994 INFO:teuthology.orchestra.run.smithi079.stdout:Package ceph-base-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 is already installed. 2023-10-30T06:18:57.044 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:18:57.045 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-10-30T06:18:57.045 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:18:57.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mgr-debuginfo 2023-10-30T06:18:57.440 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:18:57.539 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:18:57.685 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:06:53 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:18:57.825 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:18:57.826 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:18:57.826 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:18:57.827 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:18:57.827 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:18:57.827 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 19 M 2023-10-30T06:18:57.827 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:18:57.827 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:18:57.828 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:18:57.828 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:18:57.828 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:18:57.828 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 19 M 2023-10-30T06:18:57.828 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 101 M 2023-10-30T06:18:57.829 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:18:59.167 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-mgr-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 14 MB/s | 19 MB 00:01 2023-10-30T06:18:59.168 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:18:59.169 INFO:teuthology.orchestra.run.smithi079.stdout:Total 14 MB/s | 19 MB 00:01 2023-10-30T06:18:59.169 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:18:59.172 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:18:59.172 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:18:59.318 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:18:59.318 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:19:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:01.066 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:01.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:19:01.067 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:19:01.067 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:19:01.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:19:01.068 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:19:01.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-osd-debuginfo 2023-10-30T06:19:01.505 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:19:01.622 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:01.772 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:06:58 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:19:01.909 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:19:01.909 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:19:01.910 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:19:01.910 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:19:01.910 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:19:01.910 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 315 M 2023-10-30T06:19:01.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:19:01.911 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:19:01.911 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:19:01.911 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:19:01.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:19:01.912 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 315 M 2023-10-30T06:19:01.912 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 1.4 G 2023-10-30T06:19:01.912 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:19:02.570 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:02.829 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:02.829 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:02.830 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:19:02.830 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:19:02.830 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:02.830 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:19:03.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mds-debuginfo 2023-10-30T06:19:03.584 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:06:59 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:19:03.733 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:19:03.734 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:03.734 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:19:03.735 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:03.735 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:19:03.735 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 47 M 2023-10-30T06:19:03.735 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 47 M 2023-10-30T06:19:03.736 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 170 M 2023-10-30T06:19:03.737 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:19:17.252 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-mds-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 3.5 MB/s | 47 MB 00:13 2023-10-30T06:19:17.252 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:19:17.253 INFO:teuthology.orchestra.run.smithi079.stdout:Total 3.5 MB/s | 47 MB 00:13 2023-10-30T06:19:17.253 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:19:17.256 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:19:17.257 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:19:17.568 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:19:17.568 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:19:19.283 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-osd-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 18 MB/s | 315 MB 00:17 2023-10-30T06:19:19.284 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:19:19.285 INFO:teuthology.orchestra.run.smithi112.stdout:Total 18 MB/s | 315 MB 00:17 2023-10-30T06:19:19.285 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:19:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:19:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:19:21.172 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:19:21.172 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:19:22.237 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:19:22.340 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:23.725 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:23.983 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:23.984 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:23.984 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:19:23.984 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:19:23.984 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:23.984 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:19:24.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-mon-debuginfo 2023-10-30T06:19:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:07:20 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:19:24.877 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:19:24.878 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:19:24.878 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:24.878 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 112 M 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:19:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:24.880 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 112 M 2023-10-30T06:19:24.880 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 509 M 2023-10-30T06:19:24.880 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:19:30.470 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-mon-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 20 MB/s | 112 MB 00:05 2023-10-30T06:19:30.471 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:19:30.471 INFO:teuthology.orchestra.run.smithi079.stdout:Total 20 MB/s | 112 MB 00:05 2023-10-30T06:19:30.471 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:19:30.477 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:19:30.478 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:19:31.168 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:19:31.169 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:19:42.836 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:19:43.007 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:46.344 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:19:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:46.638 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:19:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:19:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:46.638 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:19:46.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-osd-debuginfo 2023-10-30T06:19:47.358 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:07:43 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:19:47.496 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:19:47.497 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:19:47.497 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:47.497 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:19:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 315 M 2023-10-30T06:19:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:47.498 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:19:47.498 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:19:47.498 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:19:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:19:47.499 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 315 M 2023-10-30T06:19:47.499 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.4 G 2023-10-30T06:19:47.499 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:19:53.501 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:19:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:20:01.756 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:20:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:20:02.024 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:02.025 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:20:02.025 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:20:02.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:02.025 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:20:02.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-fuse-debuginfo 2023-10-30T06:20:02.761 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:07:59 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:20:02.898 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:20:02.899 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:20:02.899 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:02.899 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.8 M 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:20:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 5.8 M 2023-10-30T06:20:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 28 M 2023-10-30T06:20:02.901 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:20:05.408 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-fuse-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 2.3 MB/s | 5.8 MB 00:02 2023-10-30T06:20:05.409 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:05.409 INFO:teuthology.orchestra.run.smithi112.stdout:Total 2.3 MB/s | 5.8 MB 00:02 2023-10-30T06:20:05.410 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:20:05.414 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:20:05.415 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:20:05.484 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:20:05.484 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:20:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:20:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:06.908 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:07.166 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:20:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:20:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:07.167 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:20:07.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados-devel-debuginfo 2023-10-30T06:20:07.858 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:08:04 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:20:07.996 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:20:07.998 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:07.998 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:20:07.998 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:07.999 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:20:07.999 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 108 k 2023-10-30T06:20:08.000 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:08.000 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:20:08.000 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:08.001 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:20:08.001 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:08.001 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 108 k 2023-10-30T06:20:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 440 k 2023-10-30T06:20:08.002 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:20:08.452 INFO:teuthology.orchestra.run.smithi112.stdout:librados-devel-debuginfo-18.0.0-6858.gd0db9f2f. 238 kB/s | 108 kB 00:00 2023-10-30T06:20:08.453 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:08.453 INFO:teuthology.orchestra.run.smithi112.stdout:Total 238 kB/s | 108 kB 00:00 2023-10-30T06:20:08.453 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:20:08.456 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:20:08.456 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:20:08.489 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:20:08.490 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:20:08.605 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:20:08.725 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:20:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:20:09.362 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:20:09.362 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:09.363 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:20:09.363 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:20:09.363 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:09.363 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:20:09.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install libcephfs2-debuginfo 2023-10-30T06:20:10.058 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:08:07 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:20:10.197 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:20:10.198 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:10.198 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:20:10.199 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:10.199 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:20:10.199 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.1 M 2023-10-30T06:20:10.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:10.200 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:20:10.200 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:10.200 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:20:10.200 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:10.201 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 5.1 M 2023-10-30T06:20:10.201 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 24 M 2023-10-30T06:20:10.201 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:20:11.538 INFO:teuthology.orchestra.run.smithi112.stdout:libcephfs2-debuginfo-18.0.0-6858.gd0db9f2f.el9. 3.8 MB/s | 5.1 MB 00:01 2023-10-30T06:20:11.542 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:11.542 INFO:teuthology.orchestra.run.smithi112.stdout:Total 3.8 MB/s | 5.1 MB 00:01 2023-10-30T06:20:11.542 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:20:11.543 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:20:11.543 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:20:11.607 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:20:11.607 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:20:12.246 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:20:12.352 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:20:12.870 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:20:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:20:13.130 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:13.130 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:20:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:20:13.131 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:13.131 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:20:13.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librados2-debuginfo 2023-10-30T06:20:13.846 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:08:10 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:20:13.983 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:20:13.984 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:20:13.985 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:13.985 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:20:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 45 M 2023-10-30T06:20:13.985 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:13.986 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:20:13.986 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:13.986 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:20:13.986 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:13.986 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 45 M 2023-10-30T06:20:13.987 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 256 M 2023-10-30T06:20:13.987 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:20:19.749 INFO:teuthology.orchestra.run.smithi112.stdout:librados2-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 7.9 MB/s | 45 MB 00:05 2023-10-30T06:20:19.750 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:19.751 INFO:teuthology.orchestra.run.smithi112.stdout:Total 7.9 MB/s | 45 MB 00:05 2023-10-30T06:20:19.751 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:20:19.754 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:20:19.755 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:20:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:20:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:20:22.748 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-osd-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 8.9 MB/s | 315 MB 00:35 2023-10-30T06:20:22.749 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:22.749 INFO:teuthology.orchestra.run.smithi079.stdout:Total 8.9 MB/s | 315 MB 00:35 2023-10-30T06:20:22.750 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:20:22.753 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:20:22.754 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:20:24.651 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:20:24.651 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:20:25.202 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:20:25.306 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:27.242 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:20:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:27.503 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:20:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:20:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:27.504 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:20:27.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install librbd1-debuginfo 2023-10-30T06:20:28.215 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:08:25 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:20:28.352 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:20:28.353 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:20:28.354 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:28.354 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:20:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 51 M 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:20:28.355 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 51 M 2023-10-30T06:20:28.356 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 298 M 2023-10-30T06:20:28.356 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:20:55.458 INFO:teuthology.orchestra.run.smithi112.stdout:librbd1-debuginfo-18.0.0-6858.gd0db9f2f.el9.x86 1.9 MB/s | 51 MB 00:27 2023-10-30T06:20:55.459 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:20:55.459 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.9 MB/s | 51 MB 00:27 2023-10-30T06:20:55.459 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:20:55.463 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:20:55.463 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:20:55.804 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:20:55.805 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:20:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:20:56.957 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:01.686 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:01.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:03.833 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:04.101 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:04.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:04.102 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:04.102 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:04.102 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:04.103 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:04.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-cephfs-debuginfo 2023-10-30T06:21:04.866 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:01 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:05.005 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:05.006 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:05.007 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:05.007 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 615 k 2023-10-30T06:21:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:05.007 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:05.008 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:05.008 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:05.008 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 615 k 2023-10-30T06:21:05.008 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.1 M 2023-10-30T06:21:05.009 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:05.485 INFO:teuthology.orchestra.run.smithi112.stdout:python3-cephfs-debuginfo-18.0.0-6858.gd0db9f2f. 1.3 MB/s | 615 kB 00:00 2023-10-30T06:21:05.486 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:05.486 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.3 MB/s | 615 kB 00:00 2023-10-30T06:21:05.486 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:05.489 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:05.489 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:05.525 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:05.526 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:05.716 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:05.976 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:05.977 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:05.977 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:05.977 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:05.977 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:05.977 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:06.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-fuse-debuginfo 2023-10-30T06:21:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:06.461 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:06.462 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:06.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rados-debuginfo 2023-10-30T06:21:06.751 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:02 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:06.889 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:06.890 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:06.890 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:06.890 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:06.890 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.8 M 2023-10-30T06:21:06.891 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:06.891 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:06.891 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:06.891 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:06.892 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:06.892 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 5.8 M 2023-10-30T06:21:06.892 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 28 M 2023-10-30T06:21:06.892 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:07.168 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:04 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:07.307 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:07.308 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:07.308 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:07.308 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 769 k 2023-10-30T06:21:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:07.309 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:07.309 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:07.309 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:07.310 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 769 k 2023-10-30T06:21:07.310 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.7 M 2023-10-30T06:21:07.310 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:07.771 INFO:teuthology.orchestra.run.smithi112.stdout:python3-rados-debuginfo-18.0.0-6858.gd0db9f2f.e 1.6 MB/s | 769 kB 00:00 2023-10-30T06:21:07.772 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:07.772 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.6 MB/s | 769 kB 00:00 2023-10-30T06:21:07.772 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:07.775 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:07.775 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:07.814 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:07.814 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:07.997 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:08.495 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:08.768 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:08.769 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:08.769 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:08.769 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:08.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rbd-debuginfo 2023-10-30T06:21:09.287 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-fuse-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 2.4 MB/s | 5.8 MB 00:02 2023-10-30T06:21:09.288 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:09.288 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.4 MB/s | 5.8 MB 00:02 2023-10-30T06:21:09.289 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:09.293 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:09.293 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:09.364 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:09.365 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:09.471 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:06 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:09.609 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:09.610 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:09.610 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:09.611 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:09.611 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:09.611 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.1 M 2023-10-30T06:21:09.611 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:09.611 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:09.612 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:09.612 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:09.612 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:09.612 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 1.1 M 2023-10-30T06:21:09.612 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 4.3 M 2023-10-30T06:21:09.613 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:10.082 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:10.174 INFO:teuthology.orchestra.run.smithi112.stdout:python3-rbd-debuginfo-18.0.0-6858.gd0db9f2f.el9 2.0 MB/s | 1.1 MB 00:00 2023-10-30T06:21:10.175 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:10.175 INFO:teuthology.orchestra.run.smithi112.stdout:Total 2.0 MB/s | 1.1 MB 00:00 2023-10-30T06:21:10.175 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:10.178 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:10.178 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:10.193 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:10.219 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:10.219 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:10.434 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:10.780 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:10.962 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:11.061 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:11.061 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:11.061 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:11.062 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:11.062 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:11.062 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:11.215 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:11.216 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:11.216 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:11.216 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:11.262 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados-devel-debuginfo 2023-10-30T06:21:11.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install python3-rgw-debuginfo 2023-10-30T06:21:11.809 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:07 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:11.948 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:11.949 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:11.949 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:11.949 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:11.949 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:11.949 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 108 k 2023-10-30T06:21:11.950 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:11.950 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:11.950 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:11.950 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:11.950 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:11.951 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 108 k 2023-10-30T06:21:11.951 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 440 k 2023-10-30T06:21:11.951 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:12.005 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:09 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:12.145 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:12.146 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:12.146 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:12.147 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:12.147 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:12.147 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 337 k 2023-10-30T06:21:12.147 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:12.147 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 337 k 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 1.0 M 2023-10-30T06:21:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:12.456 INFO:teuthology.orchestra.run.smithi079.stdout:librados-devel-debuginfo-18.0.0-6858.gd0db9f2f. 214 kB/s | 108 kB 00:00 2023-10-30T06:21:12.457 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:12.457 INFO:teuthology.orchestra.run.smithi079.stdout:Total 213 kB/s | 108 kB 00:00 2023-10-30T06:21:12.457 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:12.460 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:12.461 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:12.493 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:12.494 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:12.611 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:12.721 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:13.138 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:13.360 INFO:teuthology.orchestra.run.smithi112.stdout:python3-rgw-debuginfo-18.0.0-6858.gd0db9f2f.el9 278 kB/s | 337 kB 00:01 2023-10-30T06:21:13.361 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:13.362 INFO:teuthology.orchestra.run.smithi112.stdout:Total 277 kB/s | 337 kB 00:01 2023-10-30T06:21:13.362 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:13.365 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:13.365 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:13.405 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:13.405 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:13.417 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:13.417 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:13.417 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:13.418 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:13.418 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:13.418 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:13.548 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:13.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install libcephfs2-debuginfo 2023-10-30T06:21:13.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:14.038 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:14.154 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:10 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:14.292 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:14.293 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:14.293 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:14.294 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:14.294 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 5.1 M 2023-10-30T06:21:14.294 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:14.295 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:14.295 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:14.295 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:14.295 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:14.295 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 5.1 M 2023-10-30T06:21:14.296 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 24 M 2023-10-30T06:21:14.296 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:21:14.305 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:14.306 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:14.306 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:14.306 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:14.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-fuse-debuginfo 2023-10-30T06:21:15.017 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:12 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:15.154 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:15.155 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:15.156 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:15.156 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 506 k 2023-10-30T06:21:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:15.157 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:15.157 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:15.157 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:15.157 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 506 k 2023-10-30T06:21:15.158 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 2.5 M 2023-10-30T06:21:15.158 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:15.871 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-fuse-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 707 kB/s | 506 kB 00:00 2023-10-30T06:21:15.872 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:15.873 INFO:teuthology.orchestra.run.smithi112.stdout:Total 705 kB/s | 506 kB 00:00 2023-10-30T06:21:15.873 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:15.876 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:15.876 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:15.914 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:15.915 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:16.485 INFO:teuthology.orchestra.run.smithi079.stdout:libcephfs2-debuginfo-18.0.0-6858.gd0db9f2f.el9. 2.4 MB/s | 5.1 MB 00:02 2023-10-30T06:21:16.486 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:16.486 INFO:teuthology.orchestra.run.smithi079.stdout:Total 2.3 MB/s | 5.1 MB 00:02 2023-10-30T06:21:16.487 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:16.490 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:16.491 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:16.557 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:16.558 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:16.558 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:16.815 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:21:16.815 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:16.816 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:16.816 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:16.816 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:16.816 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:16.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-mirror-debuginfo 2023-10-30T06:21:17.201 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:17.308 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:17.524 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:14 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:17.663 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:17.664 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:17.664 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:17.664 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:17.664 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:17.665 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 51 M 2023-10-30T06:21:17.665 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:17.665 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:17.665 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:17.665 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:17.666 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:17.666 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 51 M 2023-10-30T06:21:17.666 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 289 M 2023-10-30T06:21:17.666 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:17.842 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:18.108 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:18.109 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:18.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librados2-debuginfo 2023-10-30T06:21:18.841 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:14 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:18.980 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:18.981 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:18.981 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:18.981 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:18.981 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:18.982 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 45 M 2023-10-30T06:21:18.982 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:18.982 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:18.982 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:18.982 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:18.983 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:18.983 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 45 M 2023-10-30T06:21:18.983 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 256 M 2023-10-30T06:21:18.984 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:29.769 INFO:teuthology.orchestra.run.smithi079.stdout:librados2-debuginfo-18.0.0-6858.gd0db9f2f.el9.x 4.2 MB/s | 45 MB 00:10 2023-10-30T06:21:29.770 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:29.771 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.2 MB/s | 45 MB 00:10 2023-10-30T06:21:29.771 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:29.775 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:29.776 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:30.087 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:30.088 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:32.115 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-mirror-debuginfo-18.0.0-6858.gd0db9f2f.el9. 3.5 MB/s | 51 MB 00:14 2023-10-30T06:21:32.116 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:32.116 INFO:teuthology.orchestra.run.smithi112.stdout:Total 3.5 MB/s | 51 MB 00:14 2023-10-30T06:21:32.117 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:32.120 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:32.121 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:32.453 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:32.454 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:35.228 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:35.326 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:37.265 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:37.530 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_ 1/1 2023-10-30T06:21:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:37.531 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:37.532 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:37.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install librbd1-debuginfo 2023-10-30T06:21:38.222 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:38.301 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:34 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:38.329 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:38.446 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:38.448 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:38.448 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:38.448 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 51 M 2023-10-30T06:21:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:38.449 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:38.449 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:38.449 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:38.450 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 51 M 2023-10-30T06:21:38.450 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 298 M 2023-10-30T06:21:38.450 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:40.305 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:21:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:40.578 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:40.579 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:40.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install rbd-nbd-debuginfo 2023-10-30T06:21:41.289 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:09:38 ago on Mon 30 Oct 2023 06:12:03 AM UTC. 2023-10-30T06:21:41.426 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-10-30T06:21:41.427 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:41.427 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repo Size 2023-10-30T06:21:41.427 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:41.427 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-10-30T06:21:41.427 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.2 M 2023-10-30T06:21:41.428 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:41.428 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-10-30T06:21:41.428 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-10-30T06:21:41.428 INFO:teuthology.orchestra.run.smithi112.stdout:Install 1 Package 2023-10-30T06:21:41.428 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:41.429 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 1.2 M 2023-10-30T06:21:41.429 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 5.4 M 2023-10-30T06:21:41.429 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-10-30T06:21:42.075 INFO:teuthology.orchestra.run.smithi112.stdout:rbd-nbd-debuginfo-18.0.0-6858.gd0db9f2f.el9.x86 1.9 MB/s | 1.2 MB 00:00 2023-10-30T06:21:42.076 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:42.076 INFO:teuthology.orchestra.run.smithi112.stdout:Total 1.9 MB/s | 1.2 MB 00:00 2023-10-30T06:21:42.076 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-10-30T06:21:42.079 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-10-30T06:21:42.080 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-10-30T06:21:42.122 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-10-30T06:21:42.122 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-10-30T06:21:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-10-30T06:21:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:43.128 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-10-30T06:21:43.128 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:43.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:21:43.129 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-10-30T06:21:43.322 DEBUG:teuthology.parallel:result is None 2023-10-30T06:21:45.288 INFO:teuthology.orchestra.run.smithi079.stdout:librbd1-debuginfo-18.0.0-6858.gd0db9f2f.el9.x86 7.4 MB/s | 51 MB 00:06 2023-10-30T06:21:45.288 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:45.289 INFO:teuthology.orchestra.run.smithi079.stdout:Total 7.4 MB/s | 51 MB 00:06 2023-10-30T06:21:45.289 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:45.294 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:45.294 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:45.637 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:45.638 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:53.775 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:54.042 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:21:54.042 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:54.043 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:54.043 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:54.043 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:54.043 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:54.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-cephfs-debuginfo 2023-10-30T06:21:54.788 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:50 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:54.926 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:54.927 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:54.927 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:54.927 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 615 k 2023-10-30T06:21:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:54.928 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:54.928 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:54.928 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:54.929 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 615 k 2023-10-30T06:21:54.929 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.1 M 2023-10-30T06:21:54.929 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:55.332 INFO:teuthology.orchestra.run.smithi079.stdout:python3-cephfs-debuginfo-18.0.0-6858.gd0db9f2f. 1.5 MB/s | 615 kB 00:00 2023-10-30T06:21:55.333 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:55.333 INFO:teuthology.orchestra.run.smithi079.stdout:Total 1.5 MB/s | 615 kB 00:00 2023-10-30T06:21:55.333 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:55.336 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:55.336 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:55.372 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:55.372 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:55.534 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:55.655 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:56.373 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9 1/1 2023-10-30T06:21:56.374 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:56.374 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:56.374 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:56.374 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:56.375 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:56.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rados-debuginfo 2023-10-30T06:21:57.111 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:53 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:21:57.249 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:21:57.249 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:57.250 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:21:57.250 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:57.250 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:21:57.250 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 769 k 2023-10-30T06:21:57.251 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:57.251 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:21:57.251 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:21:57.251 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:21:57.251 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:57.252 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 769 k 2023-10-30T06:21:57.252 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.7 M 2023-10-30T06:21:57.252 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:21:58.202 INFO:teuthology.orchestra.run.smithi079.stdout:python3-rados-debuginfo-18.0.0-6858.gd0db9f2f.e 808 kB/s | 769 kB 00:00 2023-10-30T06:21:58.203 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:21:58.203 INFO:teuthology.orchestra.run.smithi079.stdout:Total 807 kB/s | 769 kB 00:00 2023-10-30T06:21:58.204 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:21:58.207 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:21:58.207 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:21:58.245 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:21:58.246 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:21:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:21:58.542 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:58.963 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:59.232 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9. 1/1 2023-10-30T06:21:59.232 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:59.232 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:21:59.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:21:59.233 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:21:59.233 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:21:59.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rbd-debuginfo 2023-10-30T06:21:59.983 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:55 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:22:00.120 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:22:00.121 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:22:00.121 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:00.121 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:22:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.1 M 2023-10-30T06:22:00.122 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:00.122 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:22:00.125 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:00.125 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:22:00.126 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:00.126 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 1.1 M 2023-10-30T06:22:00.126 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 4.3 M 2023-10-30T06:22:00.126 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:22:01.664 INFO:teuthology.orchestra.run.smithi079.stdout:python3-rbd-debuginfo-18.0.0-6858.gd0db9f2f.el9 747 kB/s | 1.1 MB 00:01 2023-10-30T06:22:01.664 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:22:01.665 INFO:teuthology.orchestra.run.smithi079.stdout:Total 746 kB/s | 1.1 MB 00:01 2023-10-30T06:22:01.665 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:22:01.669 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:22:01.669 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:22:01.713 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:22:01.714 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:22:01.934 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:22:02.037 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:02.464 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:02.725 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:02.725 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:02.726 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:22:02.726 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:22:02.726 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:02.726 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:22:02.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install python3-rgw-debuginfo 2023-10-30T06:22:03.482 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:09:59 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:22:03.620 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:22:03.621 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:03.621 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:22:03.621 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:03.621 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:22:03.621 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 337 k 2023-10-30T06:22:03.622 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:03.622 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:22:03.622 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:03.622 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:22:03.622 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:03.623 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 337 k 2023-10-30T06:22:03.623 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 1.0 M 2023-10-30T06:22:03.623 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:22:04.428 INFO:teuthology.orchestra.run.smithi079.stdout:python3-rgw-debuginfo-18.0.0-6858.gd0db9f2f.el9 418 kB/s | 337 kB 00:00 2023-10-30T06:22:04.429 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:22:04.429 INFO:teuthology.orchestra.run.smithi079.stdout:Total 417 kB/s | 337 kB 00:00 2023-10-30T06:22:04.429 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:22:04.433 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:22:04.434 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:22:04.470 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:22:04.471 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:22:04.608 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:22:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x8 1/1 2023-10-30T06:22:05.391 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:05.391 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:22:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:22:05.392 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:05.392 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:22:05.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-fuse-debuginfo 2023-10-30T06:22:06.125 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:10:02 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:22:06.262 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:22:06.263 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:22:06.264 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:06.264 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:22:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 506 k 2023-10-30T06:22:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:06.264 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:22:06.265 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:06.265 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:22:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:06.265 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 506 k 2023-10-30T06:22:06.265 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 2.5 M 2023-10-30T06:22:06.266 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:22:07.114 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-fuse-debuginfo-18.0.0-6858.gd0db9f2f.el9.x8 595 kB/s | 506 kB 00:00 2023-10-30T06:22:07.115 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:22:07.115 INFO:teuthology.orchestra.run.smithi079.stdout:Total 594 kB/s | 506 kB 00:00 2023-10-30T06:22:07.115 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:22:07.118 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:22:07.118 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:22:07.155 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:22:07.156 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:22:07.310 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:22:07.436 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:22:07.826 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:22:08.114 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_6 1/1 2023-10-30T06:22:08.118 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:08.118 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:22:08.118 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:22:08.118 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:08.118 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:22:08.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-mirror-debuginfo 2023-10-30T06:22:08.850 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:10:04 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:22:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:22:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:08.993 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:22:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:22:08.994 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 51 M 2023-10-30T06:22:08.994 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:22:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:22:08.995 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 51 M 2023-10-30T06:22:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 289 M 2023-10-30T06:22:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:22:17.182 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-mirror-debuginfo-18.0.0-6858.gd0db9f2f.el9. 6.2 MB/s | 51 MB 00:08 2023-10-30T06:22:17.183 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:22:17.184 INFO:teuthology.orchestra.run.smithi079.stdout:Total 6.2 MB/s | 51 MB 00:08 2023-10-30T06:22:17.184 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:22:17.188 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:22:17.188 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:22:17.523 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:22:17.524 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:22:23.314 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:22:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:22:25.569 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:22:25.853 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86 1/1 2023-10-30T06:22:25.854 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:25.854 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:22:25.854 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:22:25.854 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:25.854 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:22:26.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install rbd-nbd-debuginfo 2023-10-30T06:22:26.586 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:10:22 ago on Mon 30 Oct 2023 06:12:04 AM UTC. 2023-10-30T06:22:26.725 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-10-30T06:22:26.726 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:26.726 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repo Size 2023-10-30T06:22:26.726 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:26.727 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-10-30T06:22:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-debuginfo x86_64 2:18.0.0-6858.gd0db9f2f.el9 ceph 1.2 M 2023-10-30T06:22:26.727 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:26.727 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-10-30T06:22:26.728 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2023-10-30T06:22:26.728 INFO:teuthology.orchestra.run.smithi079.stdout:Install 1 Package 2023-10-30T06:22:26.728 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:26.728 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 1.2 M 2023-10-30T06:22:26.728 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 5.4 M 2023-10-30T06:22:26.729 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-10-30T06:22:27.117 INFO:teuthology.orchestra.run.smithi079.stdout:rbd-nbd-debuginfo-18.0.0-6858.gd0db9f2f.el9.x86 3.2 MB/s | 1.2 MB 00:00 2023-10-30T06:22:27.118 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-10-30T06:22:27.118 INFO:teuthology.orchestra.run.smithi079.stdout:Total 3.1 MB/s | 1.2 MB 00:00 2023-10-30T06:22:27.119 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-10-30T06:22:27.121 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-10-30T06:22:27.122 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-10-30T06:22:27.162 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-10-30T06:22:27.162 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-10-30T06:22:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-10-30T06:22:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:22:27.959 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:22:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 1/1 2023-10-30T06:22:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:28.235 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-10-30T06:22:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-debuginfo-2:18.0.0-6858.gd0db9f2f.el9.x86_64 2023-10-30T06:22:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:28.236 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-10-30T06:22:28.430 DEBUG:teuthology.parallel:result is None 2023-10-30T06:22:28.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0db9f2f95b9d6344950185071cace2a50d296eb 2023-10-30T06:22:28.653 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-30T06:22:28.673 INFO:teuthology.orchestra.run.smithi079.stdout:18.0.0-6858.gd0db9f2f.el9 2023-10-30T06:22:28.674 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6858.gd0db9f2f.el9 2023-10-30T06:22:28.674 INFO:teuthology.task.install:The correct ceph version 18.0.0-6858.gd0db9f2f is installed. 2023-10-30T06:22:28.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0db9f2f95b9d6344950185071cace2a50d296eb 2023-10-30T06:22:28.892 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-30T06:22:28.917 INFO:teuthology.orchestra.run.smithi112.stdout:18.0.0-6858.gd0db9f2f.el9 2023-10-30T06:22:28.917 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6858.gd0db9f2f.el9 2023-10-30T06:22:28.917 INFO:teuthology.task.install:The correct ceph version 18.0.0-6858.gd0db9f2f is installed. 2023-10-30T06:22:28.918 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-30T06:22:28.918 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:28.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T06:22:28.946 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:28.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-30T06:22:28.987 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-30T06:22:28.987 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:28.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T06:22:29.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T06:22:29.081 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:29.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2023-10-30T06:22:29.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-30T06:22:29.180 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-30T06:22:29.180 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:29.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T06:22:29.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T06:22:29.277 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:29.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-30T06:22:29.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-30T06:22:29.374 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-30T06:22:29.375 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:29.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T06:22:29.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T06:22:29.468 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:29.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2023-10-30T06:22:29.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-30T06:22:29.564 INFO:teuthology.run_tasks:Running task ceph... 2023-10-30T06:22:29.634 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-30T06:22:29.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2023-10-30T06:22:29.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2023-10-30T06:22:29.665 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-30T06:22:29.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T06:22:29.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-30T06:22:29.732 INFO:tasks.ceph:Creating extra log directories... 2023-10-30T06:22:29.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T06:22:29.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-30T06:22:29.812 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-30T06:22:29.813 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-30T06:22:29.813 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005', 'branch': 'main', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7441005', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'}}, 'owner': 'scheduled_sjust@teuthology', 'priority': 51, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5205, 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'sleep_before_teardown': 0, 'subset': '220/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'targets': {'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOy9njApuL9486j3mpSSWwFrt0UsaVb0q/6Gn4YBG+PqZmwquojCA12rs3VU4DVsLcQDMY61NwRuS+U4/5jWhEU=', 'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6npLNQ3Gfa1m3akue2LinVUVmcPcQi3rhjOXz5RrAMVi0QhDKvV/Xf1DeWYnn3MpsGFcqGrbCo/2Pe0QXkCw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'mon_recovery': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-10-30_05:47:25', 'tube': 'smithi', 'user': 'sjust', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-10-30T06:22:29.814 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T06:22:29.838 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-30T06:22:29.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T06:22:29.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2023-10-30T06:22:29.921 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:29.921 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T06:22:29.971 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T06:22:29.972 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2023-10-30T06:22:30.026 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T06:22:30.026 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.026 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 664 Links: 1 2023-10-30T06:22:30.027 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.027 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.027 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-10-30 06:13:58.993330181 +0000 2023-10-30T06:22:30.027 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-10-30 06:11:17.095166612 +0000 2023-10-30T06:22:30.028 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-10-30 06:11:17.095166612 +0000 2023-10-30T06:22:30.028 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2023-10-30 06:11:17.095166612 +0000 2023-10-30T06:22:30.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T06:22:30.091 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-10-30T06:22:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-10-30T06:22:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000198233 s, 2.6 MB/s 2023-10-30T06:22:30.093 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T06:22:30.149 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2023-10-30T06:22:30.204 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T06:22:30.204 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.205 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-10-30T06:22:30.205 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.205 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.205 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-10-30 06:13:58.993330181 +0000 2023-10-30T06:22:30.206 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-10-30 06:11:17.578155620 +0000 2023-10-30T06:22:30.206 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-10-30 06:11:17.578155620 +0000 2023-10-30T06:22:30.206 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2023-10-30 06:11:17.578155620 +0000 2023-10-30T06:22:30.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T06:22:30.269 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-10-30T06:22:30.270 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-10-30T06:22:30.270 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000161332 s, 3.2 MB/s 2023-10-30T06:22:30.271 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T06:22:30.327 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2023-10-30T06:22:30.382 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T06:22:30.382 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.382 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-10-30T06:22:30.382 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.383 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.383 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-10-30 06:13:58.994330159 +0000 2023-10-30T06:22:30.383 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-10-30 06:11:18.074144332 +0000 2023-10-30T06:22:30.383 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-10-30 06:11:18.074144332 +0000 2023-10-30T06:22:30.383 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2023-10-30 06:11:18.074144332 +0000 2023-10-30T06:22:30.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T06:22:30.447 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-10-30T06:22:30.448 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-10-30T06:22:30.448 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00024516 s, 2.1 MB/s 2023-10-30T06:22:30.449 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T06:22:30.504 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2023-10-30T06:22:30.558 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T06:22:30.559 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.559 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-10-30T06:22:30.559 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.559 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.560 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-10-30 06:13:58.994330159 +0000 2023-10-30T06:22:30.560 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-10-30 06:11:18.547133567 +0000 2023-10-30T06:22:30.560 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-10-30 06:11:18.547133567 +0000 2023-10-30T06:22:30.560 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2023-10-30 06:11:18.547133567 +0000 2023-10-30T06:22:30.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T06:22:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-10-30T06:22:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-10-30T06:22:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000233194 s, 2.2 MB/s 2023-10-30T06:22:30.626 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T06:22:30.682 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:30.682 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:30.683 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2023-10-30T06:22:30.700 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-30T06:22:30.701 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2023-10-30T06:22:30.757 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-30T06:22:30.757 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.757 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-10-30T06:22:30.757 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.758 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.758 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-30 06:13:52.770696257 +0000 2023-10-30T06:22:30.758 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-30 06:11:17.089403497 +0000 2023-10-30T06:22:30.758 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-30 06:11:17.089403497 +0000 2023-10-30T06:22:30.759 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-10-30 06:11:17.089403497 +0000 2023-10-30T06:22:30.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-30T06:22:30.824 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-30T06:22:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-30T06:22:30.825 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000210766 s, 2.4 MB/s 2023-10-30T06:22:30.827 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-30T06:22:30.883 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2023-10-30T06:22:30.939 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-30T06:22:30.939 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:30.939 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-10-30T06:22:30.940 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:30.940 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:30.940 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-30 06:13:52.770696257 +0000 2023-10-30T06:22:30.940 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-30 06:11:17.582390137 +0000 2023-10-30T06:22:30.941 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-30 06:11:17.582390137 +0000 2023-10-30T06:22:30.941 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-10-30 06:11:17.582390137 +0000 2023-10-30T06:22:30.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-30T06:22:31.006 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-30T06:22:31.006 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-30T06:22:31.007 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000187479 s, 2.7 MB/s 2023-10-30T06:22:31.008 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-30T06:22:31.065 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2023-10-30T06:22:31.120 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-30T06:22:31.121 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:31.121 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 703 Links: 1 2023-10-30T06:22:31.121 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:31.121 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:31.121 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-30 06:13:52.770696257 +0000 2023-10-30T06:22:31.122 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-30 06:11:18.072376857 +0000 2023-10-30T06:22:31.122 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-30 06:11:18.072376857 +0000 2023-10-30T06:22:31.122 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-10-30 06:11:18.072376857 +0000 2023-10-30T06:22:31.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-30T06:22:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-30T06:22:31.186 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-30T06:22:31.187 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000240703 s, 2.1 MB/s 2023-10-30T06:22:31.188 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-30T06:22:31.244 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2023-10-30T06:22:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-30T06:22:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-30T06:22:31.299 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-10-30T06:22:31.299 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-30T06:22:31.300 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-10-30T06:22:31.300 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-10-30 06:13:52.771696230 +0000 2023-10-30T06:22:31.300 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-10-30 06:11:18.573363280 +0000 2023-10-30T06:22:31.300 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-10-30 06:11:18.573363280 +0000 2023-10-30T06:22:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-10-30 06:11:18.573363280 +0000 2023-10-30T06:22:31.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-30T06:22:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-10-30T06:22:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-10-30T06:22:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000167921 s, 3.0 MB/s 2023-10-30T06:22:31.366 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-30T06:22:31.421 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:31.422 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T06:22:31.422 INFO:tasks.ceph:Generating config... 2023-10-30T06:22:31.423 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-10-30T06:22:31.424 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-10-30T06:22:31.424 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-10-30T06:22:31.424 INFO:tasks.ceph:[global] ms type = async 2023-10-30T06:22:31.424 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2023-10-30T06:22:31.425 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-30T06:22:31.425 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-30T06:22:31.425 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-30T06:22:31.425 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-30T06:22:31.426 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-30T06:22:31.426 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-10-30T06:22:31.426 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2023-10-30T06:22:31.427 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-10-30T06:22:31.427 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-10-30T06:22:31.427 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-10-30T06:22:31.427 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-10-30T06:22:31.428 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-10-30T06:22:31.428 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-30T06:22:31.428 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2023-10-30T06:22:31.428 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-10-30T06:22:31.429 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-30T06:22:31.429 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-10-30T06:22:31.429 INFO:tasks.ceph:[osd] debug bluefs = 1 2023-10-30T06:22:31.429 INFO:tasks.ceph:[osd] debug bluestore = 1 2023-10-30T06:22:31.430 INFO:tasks.ceph:[osd] debug monc = 20 2023-10-30T06:22:31.430 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-30T06:22:31.430 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-30T06:22:31.431 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-30T06:22:31.431 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-30T06:22:31.431 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-30T06:22:31.431 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-30T06:22:31.432 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-10-30T06:22:31.432 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-10-30T06:22:31.432 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-10-30T06:22:31.432 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-30T06:22:31.433 INFO:tasks.ceph:[osd] osd fast shutdown = False 2023-10-30T06:22:31.433 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-10-30T06:22:31.433 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-10-30T06:22:31.433 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-10-30T06:22:31.434 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-30T06:22:31.434 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-10-30T06:22:31.434 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-10-30T06:22:31.434 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-30T06:22:31.435 INFO:tasks.ceph:Setting up mon.a... 2023-10-30T06:22:31.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-30T06:22:31.495 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2023-10-30T06:22:31.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-30T06:22:31.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T06:22:31.609 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.c', '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]'), ('mon.b', '172.21.15.112')] 2023-10-30T06:22:31.609 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.112', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-10-30T06:22:31.610 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:31.611 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T06:22:31.664 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --addv c '[v2:172.21.15.79:3301,v1:172.21.15.79:6790]' --add b 172.21.15.112 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:31.748 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:31.748 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid 09a24e68-832d-4fbc-9604-48e808368d81 2023-10-30T06:22:31.748 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = pacific 2023-10-30T06:22:31.748 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2023-10-30T06:22:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 09a24e68-832d-4fbc-9604-48e808368d81 2023-10-30T06:22:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2023-10-30T06:22:31.747585+0000 2023-10-30T06:22:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:created 2023-10-30T06:22:31.747585+0000 2023-10-30T06:22:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 16 (pacific) 2023-10-30T06:22:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 3 2023-10-30T06:22:31.750 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2023-10-30T06:22:31.750 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2023-10-30T06:22:31.750 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] mon.c 2023-10-30T06:22:31.750 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-30T06:22:31.751 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-30T06:22:31.806 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 09a24e68-832d-4fbc-9604-48e808368d81... 2023-10-30T06:22:31.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T06:22:31.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-30T06:22:31.895 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2023-10-30T06:22:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2023-10-30T06:22:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T06:22:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2023-10-30T06:22:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2023-10-30T06:22:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2023-10-30T06:22:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2023-10-30T06:22:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T06:22:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2023-10-30T06:22:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2023-10-30T06:22:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2023-10-30T06:22:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2023-10-30T06:22:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2023-10-30T06:22:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2023-10-30T06:22:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2023-10-30T06:22:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2023-10-30T06:22:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T06:22:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T06:22:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2023-10-30T06:22:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T06:22:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2023-10-30T06:22:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2023-10-30T06:22:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.112" 2023-10-30T06:22:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: mon client directed command retry = 5 2023-10-30T06:22:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: mon election default strategy = 3 2023-10-30T06:22:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: ms inject socket failures = 5000 2023-10-30T06:22:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: ms type = async 2023-10-30T06:22:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: osd heartbeat grace = 80 2023-10-30T06:22:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 09a24e68-832d-4fbc-9604-48e808368d81 2023-10-30T06:22:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.905 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2023-10-30T06:22:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2023-10-30T06:22:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2023-10-30T06:22:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2023-10-30T06:22:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2023-10-30T06:22:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2023-10-30T06:22:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2023-10-30T06:22:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2023-10-30T06:22:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2023-10-30T06:22:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2023-10-30T06:22:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2023-10-30T06:22:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2023-10-30T06:22:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2023-10-30T06:22:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2023-10-30T06:22:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2023-10-30T06:22:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2023-10-30T06:22:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2023-10-30T06:22:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression algorithm = zstd 2023-10-30T06:22:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore compression mode = aggressive 2023-10-30T06:22:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2023-10-30T06:22:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore zero block detection = True 2023-10-30T06:22:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 1 2023-10-30T06:22:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 1 2023-10-30T06:22:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: debug monc = 20 2023-10-30T06:22:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-10-30T06:22:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2023-10-30T06:22:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 10 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: osd blocked scrub grace period = 3600 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify cached snaps = True 2023-10-30T06:22:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify missing on start = True 2023-10-30T06:22:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2023-10-30T06:22:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: osd fast shutdown = False 2023-10-30T06:22:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: osd max markdown count = 1000 2023-10-30T06:22:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock override recovery settings = True 2023-10-30T06:22:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2023-10-30T06:22:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: osd op queue = debug_random 2023-10-30T06:22:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: osd op queue cut off = debug_random 2023-10-30T06:22:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: osd shutdown pgref assert = True 2023-10-30T06:22:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.915 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2023-10-30T06:22:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-10-30T06:22:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2023-10-30T06:22:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-10-30T06:22:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-10-30T06:22:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-10-30T06:22:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-10-30T06:22:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2023-10-30T06:22:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-10-30T06:22:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2023-10-30T06:22:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2023-10-30T06:22:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T06:22:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T06:22:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T06:22:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T06:22:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T06:22:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2023-10-30T06:22:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: mon min osdmap epochs = 50 2023-10-30T06:22:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd crush smoke test = False 2023-10-30T06:22:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: mon osdmap full prune interval = 2 2023-10-30T06:22:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: mon osdmap full prune min = 15 2023-10-30T06:22:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: mon osdmap full prune txsize = 2 2023-10-30T06:22:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: mon scrub interval = 300 2023-10-30T06:22:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: paxos service trim min = 10 2023-10-30T06:22:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:22:31.923 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2023-10-30T06:22:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2023-10-30T06:22:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2023-10-30T06:22:31.924 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2023-10-30T06:22:31.926 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2023-10-30T06:22:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2023-10-30T06:22:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-30T06:22:31.926 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2023-10-30T06:22:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2023-10-30T06:22:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2023-10-30T06:22:31.927 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2023-10-30T06:22:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2023-10-30T06:22:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2023-10-30T06:22:31.928 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2023-10-30T06:22:31.929 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-30T06:22:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2023-10-30T06:22:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2023-10-30T06:22:31.930 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2023-10-30T06:22:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.932 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2023-10-30T06:22:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2023-10-30T06:22:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2023-10-30T06:22:31.933 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.112" 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: mon client directed command retry = 5 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: mon election default strategy = 3 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: ms inject socket failures = 5000 2023-10-30T06:22:31.934 INFO:teuthology.orchestra.run.smithi112.stdout: ms type = async 2023-10-30T06:22:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: osd heartbeat grace = 80 2023-10-30T06:22:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 09a24e68-832d-4fbc-9604-48e808368d81 2023-10-30T06:22:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.935 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2023-10-30T06:22:31.935 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2023-10-30T06:22:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2023-10-30T06:22:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2023-10-30T06:22:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2023-10-30T06:22:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2023-10-30T06:22:31.937 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2023-10-30T06:22:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2023-10-30T06:22:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2023-10-30T06:22:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.938 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2023-10-30T06:22:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2023-10-30T06:22:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2023-10-30T06:22:31.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2023-10-30T06:22:31.940 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2023-10-30T06:22:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore compression algorithm = zstd 2023-10-30T06:22:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore compression mode = aggressive 2023-10-30T06:22:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2023-10-30T06:22:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore zero block detection = True 2023-10-30T06:22:31.941 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 1 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 1 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: debug monc = 20 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2023-10-30T06:22:31.942 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2023-10-30T06:22:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2023-10-30T06:22:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2023-10-30T06:22:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: osd blocked scrub grace period = 3600 2023-10-30T06:22:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify cached snaps = True 2023-10-30T06:22:31.943 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify missing on start = True 2023-10-30T06:22:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2023-10-30T06:22:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: osd fast shutdown = False 2023-10-30T06:22:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: osd max markdown count = 1000 2023-10-30T06:22:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock override recovery settings = True 2023-10-30T06:22:31.944 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: osd op queue = debug_random 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: osd op queue cut off = debug_random 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: osd shutdown pgref assert = True 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2023-10-30T06:22:31.945 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-30T06:22:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2023-10-30T06:22:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-10-30T06:22:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-10-30T06:22:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-10-30T06:22:31.946 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-10-30T06:22:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2023-10-30T06:22:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.947 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2023-10-30T06:22:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-10-30T06:22:31.947 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-10-30T06:22:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2023-10-30T06:22:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-10-30T06:22:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2023-10-30T06:22:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2023-10-30T06:22:31.948 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-30T06:22:31.949 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2023-10-30T06:22:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2023-10-30T06:22:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2023-10-30T06:22:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-30T06:22:31.950 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2023-10-30T06:22:31.951 INFO:teuthology.orchestra.run.smithi112.stdout: mon min osdmap epochs = 50 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd crush smoke test = False 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: mon osdmap full prune interval = 2 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: mon osdmap full prune min = 15 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: mon osdmap full prune txsize = 2 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: mon scrub interval = 300 2023-10-30T06:22:31.952 INFO:teuthology.orchestra.run.smithi112.stdout: paxos service trim min = 10 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2023-10-30T06:22:31.953 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-30T06:22:31.954 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-30T06:22:31.954 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2023-10-30T06:22:31.954 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2023-10-30T06:22:31.954 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2023-10-30T06:22:31.955 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-30T06:22:31.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-30T06:22:32.021 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-30T06:22:32.022 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:32.022 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-30T06:22:32.037 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:32.037 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-30T06:22:32.092 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:22:32.092 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:32.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T06:22:32.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T06:22:32.171 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:22:32.171 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:32.226 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:22:32.226 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:32.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-30T06:22:32.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-30T06:22:32.267 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:22:32.267 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:32.322 INFO:tasks.ceph:Setting up mon nodes... 2023-10-30T06:22:32.323 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-30T06:22:32.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T06:22:32.377 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-30T06:22:32.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T06:22:32.445 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-30T06:22:32.448 INFO:tasks.ceph:Setting up mds nodes... 2023-10-30T06:22:32.448 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-30T06:22:32.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-30T06:22:32.490 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-30T06:22:32.506 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-30T06:22:32.549 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-30T06:22:32.563 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-30T06:22:32.564 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-30T06:22:32.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-30T06:22:32.591 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:32.592 INFO:tasks.ceph:role: osd.0 2023-10-30T06:22:32.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:22:32.592 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T06:22:32.665 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:32.665 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:32.666 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:32.666 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:32.666 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:32.666 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:32.667 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:33.170 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-10-30T06:22:33.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-10-30T06:22:33.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-30T06:22:33.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-30T06:22:33.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-30T06:22:33.403 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:33.404 INFO:tasks.ceph:role: osd.1 2023-10-30T06:22:33.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:22:33.404 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T06:22:33.477 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:33.478 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:33.478 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:33.478 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:33.478 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:33.478 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:33.479 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:33.479 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:33.479 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:33.479 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:33.939 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-10-30T06:22:33.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-10-30T06:22:33.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-30T06:22:34.034 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-30T06:22:34.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-30T06:22:34.134 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:34.134 INFO:tasks.ceph:role: osd.2 2023-10-30T06:22:34.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:22:34.135 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T06:22:34.208 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:34.209 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:34.209 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:34.209 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:34.209 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:34.210 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:34.210 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:34.210 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:34.210 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:34.210 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:34.696 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-10-30T06:22:34.698 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-10-30T06:22:34.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-30T06:22:34.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-30T06:22:34.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-30T06:22:34.948 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:34.948 INFO:tasks.ceph:role: osd.3 2023-10-30T06:22:34.949 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:22:34.949 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T06:22:35.022 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:35.023 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:35.023 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:35.024 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:35.024 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:35.024 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:35.024 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:35.526 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-10-30T06:22:35.527 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-10-30T06:22:35.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-30T06:22:35.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-30T06:22:35.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:35.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.822+0000 7f5cf600e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-30T06:22:35.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.823+0000 7f5cf600e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-30T06:22:35.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.823+0000 7f5cf600e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T06:22:35.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.823+0000 7f5cf600e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T06:22:35.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.823+0000 7f5cf600e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-30T06:22:35.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:35.823+0000 7f5cf600e500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-30T06:22:38.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-30T06:22:38.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:38.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.787+0000 7f915a009500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-30T06:22:38.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.788+0000 7f915a009500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-30T06:22:38.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.788+0000 7f915a009500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T06:22:38.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.788+0000 7f915a009500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T06:22:38.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.788+0000 7f915a009500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-30T06:22:38.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:38.788+0000 7f915a009500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-30T06:22:41.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-30T06:22:41.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:41.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.770+0000 7f485c48c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-30T06:22:41.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.770+0000 7f485c48c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-30T06:22:41.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.770+0000 7f485c48c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T06:22:41.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.770+0000 7f485c48c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T06:22:41.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.770+0000 7f485c48c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-30T06:22:41.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:41.771+0000 7f485c48c500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-30T06:22:44.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-30T06:22:44.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.742+0000 7fae3bf4a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-30T06:22:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.743+0000 7fae3bf4a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-30T06:22:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.743+0000 7fae3bf4a500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T06:22:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.743+0000 7fae3bf4a500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T06:22:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.743+0000 7fae3bf4a500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-30T06:22:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:22:44.743+0000 7fae3bf4a500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-30T06:22:47.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-30T06:22:47.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-30T06:22:47.648 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:47.648 INFO:tasks.ceph:role: osd.4 2023-10-30T06:22:47.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:22:47.649 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-30T06:22:47.726 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:47.728 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:47.728 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:47.728 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:47.729 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:48.240 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-10-30T06:22:48.243 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-30T06:22:48.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-30T06:22:48.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-30T06:22:48.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-30T06:22:48.500 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:48.500 INFO:tasks.ceph:role: osd.5 2023-10-30T06:22:48.500 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:22:48.501 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-30T06:22:48.575 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:48.576 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:48.577 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:48.577 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:48.577 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:49.030 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-10-30T06:22:49.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-30T06:22:49.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-30T06:22:49.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-30T06:22:49.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-30T06:22:49.233 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:49.233 INFO:tasks.ceph:role: osd.6 2023-10-30T06:22:49.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:22:49.234 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-30T06:22:49.309 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:49.309 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:49.309 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:49.309 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:49.310 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:49.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:49.311 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:49.809 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-10-30T06:22:49.811 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-30T06:22:49.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-30T06:22:49.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-30T06:22:49.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-30T06:22:50.047 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-30T06:22:50.048 INFO:tasks.ceph:role: osd.7 2023-10-30T06:22:50.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:22:50.048 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-30T06:22:50.124 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-30T06:22:50.125 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-30T06:22:50.125 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-30T06:22:50.125 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-30T06:22:50.125 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-30T06:22:50.125 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-10-30T06:22:50.126 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-30T06:22:50.126 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-30T06:22:50.126 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-30T06:22:50.126 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-30T06:22:50.656 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-10-30T06:22:50.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-10-30T06:22:50.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-30T06:22:50.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-30T06:22:50.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:50.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.942+0000 7ff67a225500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-30T06:22:50.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.942+0000 7ff67a225500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-30T06:22:50.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.942+0000 7ff67a225500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T06:22:50.944 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.942+0000 7ff67a225500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T06:22:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.942+0000 7ff67a225500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-30T06:22:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:50.943+0000 7ff67a225500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-30T06:22:53.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-30T06:22:53.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.909+0000 7fb6ededa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-30T06:22:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.909+0000 7fb6ededa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-30T06:22:53.911 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.909+0000 7fb6ededa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T06:22:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.909+0000 7fb6ededa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T06:22:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.909+0000 7fb6ededa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-30T06:22:53.912 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:53.910+0000 7fb6ededa500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-30T06:22:56.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-30T06:22:56.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-30T06:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-30T06:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T06:22:56.864 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T06:22:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-30T06:22:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:56.863+0000 7ff79700e500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-30T06:22:59.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-30T06:22:59.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:22:59.823 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.821+0000 7f623a522500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-30T06:22:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.822+0000 7f623a522500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-30T06:22:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.822+0000 7f623a522500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T06:22:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.822+0000 7f623a522500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T06:22:59.824 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.822+0000 7f623a522500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-30T06:22:59.825 INFO:teuthology.orchestra.run.smithi112.stderr:2023-10-30T06:22:59.822+0000 7f623a522500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-30T06:23:02.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-30T06:23:02.694 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-30T06:23:02.694 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:02.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-30T06:23:02.726 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:02.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-30T06:23:02.797 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:02.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-30T06:23:02.867 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:02.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-30T06:23:02.936 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:02.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-30T06:23:03.003 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-30T06:23:03.035 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-30T06:23:03.105 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-30T06:23:03.174 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-30T06:23:03.242 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.243 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-30T06:23:03.310 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:03.311 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-30T06:23:03.327 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:03.327 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-30T06:23:03.366 INFO:tasks.ceph:Adding keys to all mons... 2023-10-30T06:23:03.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T06:23:03.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-30T06:23:03.409 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2023-10-30T06:23:03.410 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCoSz9lCg92FhAAruRQaMJPX8XZyXMkoieTTQ== 2023-10-30T06:23:03.410 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2023-10-30T06:23:03.410 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCrSz9lEx0eMRAArQjZUxpAgNObLA3eMw0Igw== 2023-10-30T06:23:03.410 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2023-10-30T06:23:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCuSz9l0SgHLxAAHKIkGqgXlnDxBbddlrCmRg== 2023-10-30T06:23:03.411 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2023-10-30T06:23:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCxSz9lKWj8LRAAiocAZGpIER0SFVlDmtl1Fw== 2023-10-30T06:23:03.411 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2023-10-30T06:23:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQC0Sz9lRtlYLBAAEu+qMULKnZVp86hKISXKTQ== 2023-10-30T06:23:03.412 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2023-10-30T06:23:03.412 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCoSz9lwkaMGhAAA4r725jMzpqdsKYFaYUImA== 2023-10-30T06:23:03.412 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2023-10-30T06:23:03.412 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQC6Sz9l7Wo8OBAAYylmyzH+RXc8XYHBXiUUOw== 2023-10-30T06:23:03.412 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2023-10-30T06:23:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQC9Sz9lVlJFNhAAfzAmXETwM2gZRmUBC1Wpbw== 2023-10-30T06:23:03.413 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2023-10-30T06:23:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDASz9l31F0MxAASs+YGHeaA0zgr6Sc/x3ypA== 2023-10-30T06:23:03.413 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2023-10-30T06:23:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDDSz9laBUNMRAA1x4eO54zfmwaI0AVgmXfJQ== 2023-10-30T06:23:03.414 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2023-10-30T06:23:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCoSz9lstE/HRAANpkxj/ao8dM5nydwIN1ypA== 2023-10-30T06:23:03.414 INFO:teuthology.orchestra.run.smithi079.stdout:[client.1] 2023-10-30T06:23:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQCoSz9lN/izIBAAoJ6zZRO7jUF9fha+dF7PfA== 2023-10-30T06:23:03.431 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2023-10-30T06:23:03.432 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCoSz9lCg92FhAAruRQaMJPX8XZyXMkoieTTQ== 2023-10-30T06:23:03.432 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2023-10-30T06:23:03.432 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCrSz9lEx0eMRAArQjZUxpAgNObLA3eMw0Igw== 2023-10-30T06:23:03.432 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2023-10-30T06:23:03.432 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCuSz9l0SgHLxAAHKIkGqgXlnDxBbddlrCmRg== 2023-10-30T06:23:03.433 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2023-10-30T06:23:03.433 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCxSz9lKWj8LRAAiocAZGpIER0SFVlDmtl1Fw== 2023-10-30T06:23:03.433 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2023-10-30T06:23:03.433 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC0Sz9lRtlYLBAAEu+qMULKnZVp86hKISXKTQ== 2023-10-30T06:23:03.433 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2023-10-30T06:23:03.434 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCoSz9lwkaMGhAAA4r725jMzpqdsKYFaYUImA== 2023-10-30T06:23:03.434 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2023-10-30T06:23:03.434 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC6Sz9l7Wo8OBAAYylmyzH+RXc8XYHBXiUUOw== 2023-10-30T06:23:03.434 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2023-10-30T06:23:03.434 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC9Sz9lVlJFNhAAfzAmXETwM2gZRmUBC1Wpbw== 2023-10-30T06:23:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2023-10-30T06:23:03.435 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDASz9l31F0MxAASs+YGHeaA0zgr6Sc/x3ypA== 2023-10-30T06:23:03.435 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2023-10-30T06:23:03.435 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDDSz9laBUNMRAA1x4eO54zfmwaI0AVgmXfJQ== 2023-10-30T06:23:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-10-30T06:23:03.436 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCoSz9lstE/HRAANpkxj/ao8dM5nydwIN1ypA== 2023-10-30T06:23:03.436 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2023-10-30T06:23:03.436 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCoSz9lN/izIBAAoJ6zZRO7jUF9fha+dF7PfA== 2023-10-30T06:23:03.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T06:23:03.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T06:23:03.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T06:23:03.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-30T06:23:03.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:03.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-30T06:23:04.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T06:23:04.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T06:23:04.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T06:23:04.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-30T06:23:04.121 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-30T06:23:04.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-30T06:23:04.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T06:23:04.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-30T06:23:04.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-30T06:23:04.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T06:23:04.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-30T06:23:04.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-30T06:23:04.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-30T06:23:04.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-30T06:23:04.610 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:23:04.612 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-30T06:23:04.664 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-30T06:23:04.665 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-30T06:23:04.665 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T06:23:04.666 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-10-30T06:23:04.668 INFO:tasks.ceph.mon.a:Started 2023-10-30T06:23:04.669 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-30T06:23:04.669 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T06:23:04.669 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-30T06:23:04.671 INFO:tasks.ceph.mon.c:Started 2023-10-30T06:23:04.672 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-10-30T06:23:04.672 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T06:23:04.672 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-30T06:23:04.706 INFO:tasks.ceph.mon.b:Started 2023-10-30T06:23:04.707 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-30T06:23:04.707 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-30T06:23:04.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2023-10-30T06:23:04.710 INFO:tasks.ceph.mgr.y:Started 2023-10-30T06:23:04.710 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-30T06:23:04.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2023-10-30T06:23:04.713 INFO:tasks.ceph.mgr.x:Started 2023-10-30T06:23:04.713 INFO:tasks.ceph:Setting crush tunables to default 2023-10-30T06:23:04.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2023-10-30T06:23:12.404 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:07.590 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:12.649 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:07.851 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:12.650 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:07.852 86020== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:12.715 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:07.902 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:12.961 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.163 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:12.962 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.165 86020== Warning: unimplemented fcntl command: 1036 2023-10-30T06:23:24.742 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2023-10-30T06:23:24.754 INFO:tasks.ceph:check_enable_crimson: False 2023-10-30T06:23:24.754 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-30T06:23:24.755 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:24.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-30T06:23:24.784 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:24.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-30T06:23:24.850 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:24.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-30T06:23:24.917 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-10-30T06:23:24.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-30T06:23:24.987 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:24.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-30T06:23:24.991 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:24.990+0000 7fe542a10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T06:23:25.015 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:25.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-30T06:23:25.018 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.016+0000 7fca38243040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T06:23:25.059 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.057+0000 7fe542a10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T06:23:25.074 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.073+0000 7fca38243040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T06:23:25.082 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:25.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-30T06:23:25.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-10-30T06:23:25.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-30T06:23:25.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 6e79cc30-6aa7-489d-8806-48c107644b8b 0 2023-10-30T06:23:25.595 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.594+0000 7fca38243040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T06:23:25.606 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.605+0000 7fe542a10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T06:23:25.646 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.644+0000 7fca38243040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T06:23:25.658 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.657+0000 7fe542a10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T06:23:25.690 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.688+0000 7fca38243040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T06:23:25.703 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.701+0000 7fe542a10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T06:23:25.744 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.743+0000 7fca38243040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T06:23:25.755 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.754+0000 7fe542a10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T06:23:25.790 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.789+0000 7fca38243040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T06:23:25.797 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.796+0000 7fe542a10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T06:23:25.824 INFO:teuthology.orchestra.run.smithi112.stdout:0 2023-10-30T06:23:25.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new c8a16891-3cdc-487f-9fdc-1b88ce43caf4 1 2023-10-30T06:23:25.893 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.891+0000 7fca38243040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-30T06:23:25.904 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.903+0000 7fe542a10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-30T06:23:25.964 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:25.963+0000 7fca38243040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:25.978 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:25.977+0000 7fe542a10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:26.014 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.013+0000 7fca38243040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T06:23:26.024 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.023+0000 7fe542a10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T06:23:26.067 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.066+0000 7fca38243040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T06:23:26.081 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.080+0000 7fe542a10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T06:23:26.114 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.113+0000 7fca38243040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T06:23:26.129 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.128+0000 7fe542a10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T06:23:26.218 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-10-30T06:23:26.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 0b1c9a57-cced-4163-a178-f461fc9596aa 2 2023-10-30T06:23:26.501 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.499+0000 7fca38243040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-30T06:23:26.516 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.515+0000 7fe542a10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-30T06:23:26.607 INFO:teuthology.orchestra.run.smithi112.stdout:2 2023-10-30T06:23:26.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new eab98798-a4c8-4e5f-83ec-40b3f31a4d50 3 2023-10-30T06:23:26.642 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.641+0000 7fca38243040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T06:23:26.657 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.655+0000 7fe542a10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T06:23:26.770 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.768+0000 7fca38243040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T06:23:26.787 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.786+0000 7fe542a10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T06:23:26.814 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.812+0000 7fca38243040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T06:23:26.832 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.831+0000 7fe542a10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T06:23:26.909 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:26.907+0000 7fca38243040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:26.930 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:26.929+0000 7fe542a10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:26.971 INFO:teuthology.orchestra.run.smithi112.stdout:3 2023-10-30T06:23:26.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 6f17fdbb-5762-457a-8b89-4c5a4f2197b2 4 2023-10-30T06:23:27.013 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:27.010+0000 7fca38243040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T06:23:27.029 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:27.027+0000 7fe542a10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T06:23:27.157 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:27.156+0000 7fca38243040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:27.176 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:27.174+0000 7fe542a10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:27.200 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:27.198+0000 7fca38243040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:27.217 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:27.216+0000 7fe542a10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:27.340 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:27.339+0000 7fca38243040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T06:23:27.360 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:27.359+0000 7fe542a10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T06:23:27.363 INFO:teuthology.orchestra.run.smithi112.stdout:4 2023-10-30T06:23:27.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 49ea8aae-4b05-43c9-979a-99026ae58a89 5 2023-10-30T06:23:27.914 INFO:teuthology.orchestra.run.smithi112.stdout:5 2023-10-30T06:23:27.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new ebb325d3-e1fc-44b6-929b-b7fdd71f82b1 6 2023-10-30T06:23:28.156 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:28.154+0000 7fca38243040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T06:23:28.176 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:28.174+0000 7fe542a10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T06:23:28.344 INFO:teuthology.orchestra.run.smithi112.stdout:6 2023-10-30T06:23:28.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 31813e1b-e740-49b2-9173-85911e32f3d9 7 2023-10-30T06:23:28.437 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:28.436+0000 7fca38243040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T06:23:28.454 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:28.453+0000 7fe542a10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T06:23:28.490 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:28.488+0000 7fca38243040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T06:23:28.512 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:28.511+0000 7fe542a10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T06:23:28.593 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:28.591+0000 7fca38243040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T06:23:28.613 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-10-30T06:23:28.611+0000 7fe542a10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T06:23:28.960 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:28.958+0000 7fca32b00640 -1 mgr handle_mgr_map I was active but no longer am 2023-10-30T06:23:29.137 INFO:teuthology.orchestra.run.smithi112.stdout:7 2023-10-30T06:23:29.149 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.149 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-30T06:23:29.150 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2023-10-30T06:23:29.152 INFO:tasks.ceph.osd.0:Started 2023-10-30T06:23:29.153 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.153 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-30T06:23:29.153 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2023-10-30T06:23:29.155 INFO:tasks.ceph.osd.1:Started 2023-10-30T06:23:29.156 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.156 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-30T06:23:29.156 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2023-10-30T06:23:29.159 INFO:tasks.ceph.osd.2:Started 2023-10-30T06:23:29.159 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.159 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-30T06:23:29.160 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2023-10-30T06:23:29.162 INFO:tasks.ceph.osd.3:Started 2023-10-30T06:23:29.162 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.163 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-30T06:23:29.163 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2023-10-30T06:23:29.166 INFO:tasks.ceph.osd.4:Started 2023-10-30T06:23:29.166 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.166 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-30T06:23:29.167 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2023-10-30T06:23:29.169 INFO:tasks.ceph.osd.5:Started 2023-10-30T06:23:29.169 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.169 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-30T06:23:29.170 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2023-10-30T06:23:29.172 INFO:tasks.ceph.osd.6:Started 2023-10-30T06:23:29.172 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-10-30T06:23:29.173 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-30T06:23:29.173 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-10-30T06:23:29.175 INFO:tasks.ceph.osd.7:Started 2023-10-30T06:23:29.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:23:29.480 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:29.478+0000 7fd57543a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-30T06:23:29.528 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:29.526+0000 7fd57543a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-30T06:23:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:23:29.798 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:28.674155+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:23:29.817 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-30T06:23:29.818 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-30T06:23:30.219 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.217+0000 7fd57543a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-30T06:23:30.294 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.293+0000 7fd57543a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-30T06:23:30.366 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.365+0000 7fd57543a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-30T06:23:30.458 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.455+0000 7fd57543a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-30T06:23:30.529 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.528+0000 7fd57543a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-30T06:23:30.622 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:23:30.709 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.708+0000 7fd57543a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-30T06:23:30.807 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.806+0000 7fd57543a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:30.884 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.882+0000 7fd57543a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-30T06:23:30.969 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:30.967+0000 7fd57543a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-30T06:23:31.043 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:31.042+0000 7fd57543a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-30T06:23:31.177 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:23:31.177 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:28.674155+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:23:31.193 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:23:31.600 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:31.599+0000 7fd57543a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-30T06:23:31.827 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:31.826+0000 7fd57543a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-30T06:23:32.008 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.007+0000 7fd57543a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-30T06:23:32.090 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.088+0000 7fd57543a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-30T06:23:32.282 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.281+0000 7fd57543a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:32.449 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.448+0000 7fd57543a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-30T06:23:32.695 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.694+0000 7fd57543a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-30T06:23:32.767 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.765+0000 7fd57543a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-30T06:23:32.956 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:32.955+0000 7fd57543a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-30T06:23:34.006 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:34.005+0000 7fd57543a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-30T06:23:34.461 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:34.459+0000 7fd57543a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-30T06:23:34.544 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:34.543+0000 7fd57543a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-30T06:23:34.731 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:34.729+0000 7fd57543a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-30T06:23:36.790 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:36.788+0000 7fd51684a640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-30T06:23:36.790 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:23:36.788+0000 7fd51684a640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-30T06:23:37.358 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:08.125 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.361 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:08.124 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.420 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:08.183 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.449 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:08.217 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.501 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:08.262 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.564 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:08.325 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.578 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:08.346 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.638 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:08.407 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.873 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:08.615 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.927 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:08.688 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.933 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:08.676 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.968 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:23:37.960+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:37.988 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:08.733 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:37.997 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:23:37.999 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:23:37.990+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.005 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:08.766 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.054 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:08.800 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.140 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:23:38.133+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.235 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:23:38.228+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.447 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:09.210 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.487 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:23:38.479+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.578 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:23:38.569+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.626 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:09.387 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.633 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:23:38.622+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:38.638 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:09.406 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.745 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:09.514 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.830 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:09.594 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:38.837 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:23:38.838 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":11,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:34.778016+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:23:38.848 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:23:39.004 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:09.766 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.008 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:09.750 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.022 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:09.790 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.131 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:09.894 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.136 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:09.904 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.182 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:09.943 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.254 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:09.999 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.295 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:10.056 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.317 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:10.085 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.385 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:10.127 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.403 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:10.166 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.429 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:10.197 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.564 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:10.325 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.587 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:10.354 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.594 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:10.355 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.635 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:10.381 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.670 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:10.433 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.680 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:10.422 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.694 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:10.462 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.743 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:10.499 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.799 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:10.555 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.828 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:10.589 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.852 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:10.619 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.887 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:10.648 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.937 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:10.700 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.951 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:10.696 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.957 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:10.699 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:39.964 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:10.732 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.106 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:10.867 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.142 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:10.909 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.167 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:10.928 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.215 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:10.979 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.234 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:10.980 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.242 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:10.984 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.251 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:11.020 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.365 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:23:40.358+0000 5dc2340 -1 Falling back to public interface 2023-10-30T06:23:40.393 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:11.154 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.436 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:11.203 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.457 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:11.218 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.506 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:11.270 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.530 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:11.276 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.539 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:11.281 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.546 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:11.315 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.683 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:11.445 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.728 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:11.496 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.745 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:11.506 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.804 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:11.567 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.837 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:11.606 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.841 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:11.587 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.856 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:11.598 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.861 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:11.618 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:40.974 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:11.736 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.018 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:11.785 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.036 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:11.797 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.127 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:11.895 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.131 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:11.876 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.153 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:11.895 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.243 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:12.000 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.302 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:12.063 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.324 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:12.088 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.356 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:12.119 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.399 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:12.144 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.422 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:12.164 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.494 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:12.255 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.526 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:12.294 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.534 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:12.290 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.547 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:12.309 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.559 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:12.327 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.573 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:12.334 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.634 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:12.403 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.666 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:12.434 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.668 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:12.413 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.802 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:12.558 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.922 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:12.663 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:41.953 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:12.694 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.066 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:12.827 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.081 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:12.837 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.097 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:12.858 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.179 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:12.924 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.211 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:12.956 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.370 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:13.126 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.657 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:13.414 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:42.951 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:13.707 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:43.265 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:14.021 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:43.849 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:14.606 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:43.906 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:14.662 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:45.655 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:23:46.466 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:23:46.467 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":11,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:34.778016+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:23:46.481 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:23:47.106 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:17.870 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.251 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:18.012 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.262 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:18.030 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.346 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:18.115 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.766 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:18.507 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.770 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:18.531 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:47.814 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:18.559 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:49.846 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:20.603 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.474 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:23.243 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.478 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:23.247 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.625 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:23.393 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.630 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:23.397 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.649 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:23.411 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.654 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:23.415 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.727 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:23.491 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.731 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:23.495 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.952 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:23.713 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:52.956 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:23.717 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:53.000 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:23.742 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:53.006 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:23.748 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:53.287 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:23:53.652 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:24.398 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:53.658 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:24.404 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:53.971 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:23:53.971 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":11,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:34.778016+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:23:53.991 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:23:55.522 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:26.278 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:55.525 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:26.282 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:56.600 INFO:tasks.ceph.osd.4.smithi112.stderr:==00:00:00:27.368 86608== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:56.872 INFO:tasks.ceph.osd.5.smithi112.stderr:==00:00:00:27.639 86609== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:56.881 INFO:tasks.ceph.osd.3.smithi079.stderr:==00:00:00:27.623 86365== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:57.062 INFO:tasks.ceph.osd.7.smithi112.stderr:==00:00:00:27.824 86611== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:57.153 INFO:tasks.ceph.osd.6.smithi112.stderr:==00:00:00:27.917 86610== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:57.453 INFO:tasks.ceph.osd.0.smithi079.stderr:==00:00:00:28.214 86362== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:57.997 INFO:tasks.ceph.osd.2.smithi079.stderr:==00:00:00:28.743 86364== Warning: unimplemented fcntl command: 1038 2023-10-30T06:23:59.529 INFO:tasks.ceph.osd.1.smithi079.stderr:==00:00:00:30.285 86363== Warning: unimplemented fcntl command: 1038 2023-10-30T06:24:00.799 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:01.475 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:24:01.476 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":11,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:23:34.778016+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:01.489 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:24:04.168 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:04.157+0000 5dc2340 -1 osd.4 0 log_to_monitors true 2023-10-30T06:24:04.510 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:04.506+0000 5dc2340 -1 osd.5 0 log_to_monitors true 2023-10-30T06:24:04.567 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:04.558+0000 5dc2340 -1 osd.7 0 log_to_monitors true 2023-10-30T06:24:05.067 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:04.993+0000 5dc2340 -1 osd.6 0 log_to_monitors true 2023-10-30T06:24:05.280 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:05.221+0000 5dc2340 -1 osd.3 0 log_to_monitors true 2023-10-30T06:24:05.765 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:05.759+0000 5dc2340 -1 osd.0 0 log_to_monitors true 2023-10-30T06:24:06.650 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:06.645+0000 5dc2340 -1 osd.2 0 log_to_monitors true 2023-10-30T06:24:07.554 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:07.547+0000 bc94640 -1 osd.4 0 waiting for initial osdmap 2023-10-30T06:24:07.561 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:07.555+0000 bc94640 -1 osd.5 0 waiting for initial osdmap 2023-10-30T06:24:07.567 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:07.562+0000 b47e640 -1 osd.7 0 waiting for initial osdmap 2023-10-30T06:24:07.934 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:07.920+0000 11a5d640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:07.948 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:07.934+0000 11a5d640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:08.013 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:08.008+0000 5dc2340 -1 osd.1 0 log_to_monitors true 2023-10-30T06:24:08.017 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:07.997+0000 1225e640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:08.296 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:08.643 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:08.638+0000 bc94640 -1 osd.6 0 waiting for initial osdmap 2023-10-30T06:24:08.677 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:08.673+0000 b47e640 -1 osd.3 0 waiting for initial osdmap 2023-10-30T06:24:08.989 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:24:08.990 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":14,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:24:08.171556+0000","last_up_change":"0.000000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:09.002 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-30T06:24:09.109 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:09.095+0000 11a5d640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:09.117 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:09.103+0000 11a5d640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:09.754 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:09.749+0000 b47e640 -1 osd.2 0 waiting for initial osdmap 2023-10-30T06:24:09.785 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:09.779+0000 bc94640 -1 osd.0 0 waiting for initial osdmap 2023-10-30T06:24:10.188 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:10.174+0000 11a5d640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:10.222 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:10.208+0000 11a5d640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:11.062 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:11.045+0000 bc94640 -1 osd.1 0 waiting for initial osdmap 2023-10-30T06:24:11.576 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:11.561+0000 13260640 -1 osd.1 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-30T06:24:15.810 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:16.483 INFO:teuthology.misc.health.smithi079.stdout: 2023-10-30T06:24:16.483 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":20,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:24:15.123361+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:16.495 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-30T06:24:16.495 INFO:tasks.ceph:Creating RBD pool 2023-10-30T06:24:16.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-30T06:24:17.575 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'rbd' created 2023-10-30T06:24:17.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-30T06:24:18.668 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-30T06:24:18.685 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-30T06:24:18.686 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-30T06:24:18.686 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-30T06:24:18.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-30T06:24:18.687 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-30T06:24:19.501 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:19.512 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":6,"active_gid":4161,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":3092786684},{"type":"v1","addr":"172.21.15.79:6801","nonce":3092786684}]},"active_addr":"172.21.15.79:6801/3092786684","active_change":"2023-10-30T06:23:34.785310+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":11,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1139465334}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":2560044989}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":265295128}]}]} 2023-10-30T06:24:19.519 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-30T06:24:19.519 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-30T06:24:19.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:20.355 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:20.355 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":23,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:24:19.574254+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:20.374 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-30T06:24:20.375 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:21.112 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:21.113 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":23,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:24:19.574254+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:21.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-30T06:24:21.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-30T06:24:21.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-30T06:24:21.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-30T06:24:21.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-30T06:24:21.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-30T06:24:21.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-30T06:24:21.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-30T06:24:21.585 INFO:teuthology.orchestra.run.smithi079.stdout:77309411333 2023-10-30T06:24:21.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:24:21.607 INFO:teuthology.orchestra.run.smithi079.stdout:68719476740 2023-10-30T06:24:21.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T06:24:21.615 INFO:teuthology.orchestra.run.smithi079.stdout:68719476740 2023-10-30T06:24:21.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T06:24:21.620 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:21.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:24:21.623 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:21.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T06:24:21.631 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:21.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:24:21.652 INFO:teuthology.orchestra.run.smithi079.stdout:68719476742 2023-10-30T06:24:21.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:24:21.817 INFO:teuthology.orchestra.run.smithi079.stdout:68719476743 2023-10-30T06:24:21.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T06:24:22.474 INFO:teuthology.orchestra.run.smithi079.stdout:64424509444 2023-10-30T06:24:22.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2023-10-30T06:24:22.570 INFO:teuthology.orchestra.run.smithi079.stdout:68719476739 2023-10-30T06:24:22.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.2 2023-10-30T06:24:22.692 INFO:teuthology.orchestra.run.smithi079.stdout:68719476739 2023-10-30T06:24:22.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.0 2023-10-30T06:24:22.731 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:22.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.3 2023-10-30T06:24:22.917 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:22.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.6 2023-10-30T06:24:23.130 INFO:teuthology.orchestra.run.smithi079.stdout:77309411332 2023-10-30T06:24:23.137 INFO:teuthology.orchestra.run.smithi079.stdout:64424509444 2023-10-30T06:24:23.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411332 for osd.1 2023-10-30T06:24:23.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.4 2023-10-30T06:24:23.196 INFO:teuthology.orchestra.run.smithi079.stdout:64424509444 2023-10-30T06:24:23.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.7 2023-10-30T06:24:23.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:24:23.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T06:24:23.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T06:24:23.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:24:23.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T06:24:24.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:24:24.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:24:24.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T06:24:24.289 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2023-10-30T06:24:24.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2023-10-30T06:24:24.335 INFO:teuthology.orchestra.run.smithi079.stdout:68719476739 2023-10-30T06:24:24.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.2 2023-10-30T06:24:24.656 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:24.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.6 2023-10-30T06:24:24.677 INFO:teuthology.orchestra.run.smithi079.stdout:68719476739 2023-10-30T06:24:24.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.0 2023-10-30T06:24:24.864 INFO:teuthology.orchestra.run.smithi079.stdout:77309411332 2023-10-30T06:24:24.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411332 for osd.1 2023-10-30T06:24:24.926 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:24.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.3 2023-10-30T06:24:24.950 INFO:teuthology.orchestra.run.smithi079.stdout:64424509445 2023-10-30T06:24:24.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-10-30T06:24:25.136 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:25.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2023-10-30T06:24:25.151 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:25.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:24:25.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T06:24:25.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T06:24:25.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T06:24:25.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:24:25.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:24:25.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:24:26.107 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:26.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2023-10-30T06:24:26.124 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.231 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:26.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476741 for osd.2 2023-10-30T06:24:26.250 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.406 INFO:teuthology.orchestra.run.smithi079.stdout:68719476743 2023-10-30T06:24:26.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.6 2023-10-30T06:24:26.424 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.515 INFO:teuthology.orchestra.run.smithi079.stdout:77309411333 2023-10-30T06:24:26.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.1 2023-10-30T06:24:26.529 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.570 INFO:teuthology.orchestra.run.smithi079.stdout:68719476741 2023-10-30T06:24:26.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476741 for osd.0 2023-10-30T06:24:26.590 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.666 INFO:teuthology.orchestra.run.smithi079.stdout:68719476743 2023-10-30T06:24:26.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.3 2023-10-30T06:24:26.682 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.751 INFO:teuthology.orchestra.run.smithi079.stdout:64424509446 2023-10-30T06:24:26.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2023-10-30T06:24:26.765 DEBUG:teuthology.parallel:result is None 2023-10-30T06:24:26.766 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T06:24:26.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:24:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:27.314 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:24:27.329 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-30T06:24:27.259604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218144,"kb_used_data":1960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756576,"statfs":{"total":773094113280,"available":772870733824,"internally_reserved":0,"allocated":2007040,"data_stored":1458048,"data_compressed":1341,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003040"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.490583+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:24:21.490583+0000","last_peered":"2023-10-30T06:24:21.490583+0000","last_clean":"2023-10-30T06:24:21.490583+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:24:21.490583+0000","last_undegraded":"2023-10-30T06:24:21.490583+0000","last_fullsized":"2023-10-30T06:24:21.490583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.343934+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:24:21.343934+0000","last_peered":"2023-10-30T06:24:21.343934+0000","last_clean":"2023-10-30T06:24:21.343934+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:24:21.343934+0000","last_undegraded":"2023-10-30T06:24:21.343934+0000","last_fullsized":"2023-10-30T06:24:21.343934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.482197+0000","last_change":"2023-10-30T06:24:20.087820+0000","last_active":"2023-10-30T06:24:21.482197+0000","last_peered":"2023-10-30T06:24:21.482197+0000","last_clean":"2023-10-30T06:24:21.482197+0000","last_became_active":"2023-10-30T06:24:20.060063+0000","last_became_peered":"2023-10-30T06:24:20.060063+0000","last_unstale":"2023-10-30T06:24:21.482197+0000","last_undegraded":"2023-10-30T06:24:21.482197+0000","last_fullsized":"2023-10-30T06:24:21.482197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:25.280973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.459454+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:24:21.459454+0000","last_peered":"2023-10-30T06:24:21.459454+0000","last_clean":"2023-10-30T06:24:21.459454+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:24:21.459454+0000","last_undegraded":"2023-10-30T06:24:21.459454+0000","last_fullsized":"2023-10-30T06:24:21.459454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.486866+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:24:21.486866+0000","last_peered":"2023-10-30T06:24:21.486866+0000","last_clean":"2023-10-30T06:24:21.486866+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:24:21.486866+0000","last_undegraded":"2023-10-30T06:24:21.486866+0000","last_fullsized":"2023-10-30T06:24:21.486866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.467833+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:24:21.467833+0000","last_peered":"2023-10-30T06:24:21.467833+0000","last_clean":"2023-10-30T06:24:21.467833+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:24:21.467833+0000","last_undegraded":"2023-10-30T06:24:21.467833+0000","last_fullsized":"2023-10-30T06:24:21.467833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.472914+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:24:21.472914+0000","last_peered":"2023-10-30T06:24:21.472914+0000","last_clean":"2023-10-30T06:24:21.472914+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:24:21.472914+0000","last_undegraded":"2023-10-30T06:24:21.472914+0000","last_fullsized":"2023-10-30T06:24:21.472914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.477449+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:24:21.477449+0000","last_peered":"2023-10-30T06:24:21.477449+0000","last_clean":"2023-10-30T06:24:21.477449+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:24:21.477449+0000","last_undegraded":"2023-10-30T06:24:21.477449+0000","last_fullsized":"2023-10-30T06:24:21.477449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'64","reported_seq":102,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-30T06:24:20.608495+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:24:20.608495+0000","last_peered":"2023-10-30T06:24:20.608495+0000","last_clean":"2023-10-30T06:24:20.608495+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:24:20.608495+0000","last_undegraded":"2023-10-30T06:24:20.608495+0000","last_fullsized":"2023-10-30T06:24:20.608495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1341,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411334,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":652,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2650000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9429999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.126999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.625999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.906000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.466999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.594999999999999}]}]},{"osd":6,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.271000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.106}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.898999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.315000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.916}]}]},{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7750000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.664000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.281000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.707000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.504000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.805}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.077999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.53700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7539999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.933}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.492999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.140999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.559999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":32.021000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.274999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.398999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.388999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.969000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.931000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1639999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.337}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.733000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.124000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.361999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.518000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":87.515000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6550000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.058}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8770000000000007}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.205}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.445}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.763999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":652,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:24:27.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:24:27.916 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:27.917 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:24:27.929 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-30T06:24:27.259604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218144,"kb_used_data":1960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756576,"statfs":{"total":773094113280,"available":772870733824,"internally_reserved":0,"allocated":2007040,"data_stored":1458048,"data_compressed":1341,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003040"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.490583+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:24:21.490583+0000","last_peered":"2023-10-30T06:24:21.490583+0000","last_clean":"2023-10-30T06:24:21.490583+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:24:21.490583+0000","last_undegraded":"2023-10-30T06:24:21.490583+0000","last_fullsized":"2023-10-30T06:24:21.490583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.343934+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:24:21.343934+0000","last_peered":"2023-10-30T06:24:21.343934+0000","last_clean":"2023-10-30T06:24:21.343934+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:24:21.343934+0000","last_undegraded":"2023-10-30T06:24:21.343934+0000","last_fullsized":"2023-10-30T06:24:21.343934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.482197+0000","last_change":"2023-10-30T06:24:20.087820+0000","last_active":"2023-10-30T06:24:21.482197+0000","last_peered":"2023-10-30T06:24:21.482197+0000","last_clean":"2023-10-30T06:24:21.482197+0000","last_became_active":"2023-10-30T06:24:20.060063+0000","last_became_peered":"2023-10-30T06:24:20.060063+0000","last_unstale":"2023-10-30T06:24:21.482197+0000","last_undegraded":"2023-10-30T06:24:21.482197+0000","last_fullsized":"2023-10-30T06:24:21.482197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:25.280973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.459454+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:24:21.459454+0000","last_peered":"2023-10-30T06:24:21.459454+0000","last_clean":"2023-10-30T06:24:21.459454+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:24:21.459454+0000","last_undegraded":"2023-10-30T06:24:21.459454+0000","last_fullsized":"2023-10-30T06:24:21.459454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.486866+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:24:21.486866+0000","last_peered":"2023-10-30T06:24:21.486866+0000","last_clean":"2023-10-30T06:24:21.486866+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:24:21.486866+0000","last_undegraded":"2023-10-30T06:24:21.486866+0000","last_fullsized":"2023-10-30T06:24:21.486866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.467833+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:24:21.467833+0000","last_peered":"2023-10-30T06:24:21.467833+0000","last_clean":"2023-10-30T06:24:21.467833+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:24:21.467833+0000","last_undegraded":"2023-10-30T06:24:21.467833+0000","last_fullsized":"2023-10-30T06:24:21.467833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.472914+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:24:21.472914+0000","last_peered":"2023-10-30T06:24:21.472914+0000","last_clean":"2023-10-30T06:24:21.472914+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:24:21.472914+0000","last_undegraded":"2023-10-30T06:24:21.472914+0000","last_fullsized":"2023-10-30T06:24:21.472914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.477449+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:24:21.477449+0000","last_peered":"2023-10-30T06:24:21.477449+0000","last_clean":"2023-10-30T06:24:21.477449+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:24:21.477449+0000","last_undegraded":"2023-10-30T06:24:21.477449+0000","last_fullsized":"2023-10-30T06:24:21.477449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"20'64","reported_seq":102,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-30T06:24:20.608495+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:24:20.608495+0000","last_peered":"2023-10-30T06:24:20.608495+0000","last_clean":"2023-10-30T06:24:20.608495+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:24:20.608495+0000","last_undegraded":"2023-10-30T06:24:20.608495+0000","last_fullsized":"2023-10-30T06:24:20.608495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1341,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411334,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":652,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2650000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9429999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.126999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.625999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.906000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.466999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.594999999999999}]}]},{"osd":6,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.271000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.106}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.898999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.315000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.916}]}]},{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7750000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.664000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.281000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.707000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.504000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.805}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.077999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.53700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7539999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.933}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.492999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.140999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.559999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":32.021000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.274999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.398999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.388999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.969000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.931000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1639999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.337}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.733000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.124000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.361999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.518000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":87.515000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6550000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.058}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8770000000000007}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.205}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.445}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.763999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":652,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:24:27.930 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T06:24:27.930 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-30T06:24:27.931 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-30T06:24:27.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-30T06:24:28.517 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:28.517 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-30T06:24:28.529 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-30T06:24:28.530 INFO:teuthology.run_tasks:Running task thrashosds... 2023-10-30T06:24:28.540 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-10-30T06:24:28.541 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'd0db9f2f95b9d6344950185071cace2a50d296eb'}} 2023-10-30T06:24:28.541 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2023-10-30T06:24:28.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-10-30T06:24:29.330 INFO:tasks.thrashosds:Beginning thrashosds... 2023-10-30T06:24:29.331 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T06:24:29.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:24:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:29.875 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:24:29.886 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-30T06:24:29.259963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218148,"kb_used_data":1960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756572,"statfs":{"total":773094113280,"available":772870729728,"internally_reserved":0,"allocated":2007040,"data_stored":1458048,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.595335"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.490583+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:24:21.490583+0000","last_peered":"2023-10-30T06:24:21.490583+0000","last_clean":"2023-10-30T06:24:21.490583+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:24:21.490583+0000","last_undegraded":"2023-10-30T06:24:21.490583+0000","last_fullsized":"2023-10-30T06:24:21.490583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.343934+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:24:21.343934+0000","last_peered":"2023-10-30T06:24:21.343934+0000","last_clean":"2023-10-30T06:24:21.343934+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:24:21.343934+0000","last_undegraded":"2023-10-30T06:24:21.343934+0000","last_fullsized":"2023-10-30T06:24:21.343934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.482197+0000","last_change":"2023-10-30T06:24:20.087820+0000","last_active":"2023-10-30T06:24:21.482197+0000","last_peered":"2023-10-30T06:24:21.482197+0000","last_clean":"2023-10-30T06:24:21.482197+0000","last_became_active":"2023-10-30T06:24:20.060063+0000","last_became_peered":"2023-10-30T06:24:20.060063+0000","last_unstale":"2023-10-30T06:24:21.482197+0000","last_undegraded":"2023-10-30T06:24:21.482197+0000","last_fullsized":"2023-10-30T06:24:21.482197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:25.280973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.459454+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:24:21.459454+0000","last_peered":"2023-10-30T06:24:21.459454+0000","last_clean":"2023-10-30T06:24:21.459454+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:24:21.459454+0000","last_undegraded":"2023-10-30T06:24:21.459454+0000","last_fullsized":"2023-10-30T06:24:21.459454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.486866+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:24:21.486866+0000","last_peered":"2023-10-30T06:24:21.486866+0000","last_clean":"2023-10-30T06:24:21.486866+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:24:21.486866+0000","last_undegraded":"2023-10-30T06:24:21.486866+0000","last_fullsized":"2023-10-30T06:24:21.486866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.467833+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:24:21.467833+0000","last_peered":"2023-10-30T06:24:21.467833+0000","last_clean":"2023-10-30T06:24:21.467833+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:24:21.467833+0000","last_undegraded":"2023-10-30T06:24:21.467833+0000","last_fullsized":"2023-10-30T06:24:21.467833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.472914+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:24:21.472914+0000","last_peered":"2023-10-30T06:24:21.472914+0000","last_clean":"2023-10-30T06:24:21.472914+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:24:21.472914+0000","last_undegraded":"2023-10-30T06:24:21.472914+0000","last_fullsized":"2023-10-30T06:24:21.472914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.477449+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:24:21.477449+0000","last_peered":"2023-10-30T06:24:21.477449+0000","last_clean":"2023-10-30T06:24:21.477449+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:24:21.477449+0000","last_undegraded":"2023-10-30T06:24:21.477449+0000","last_fullsized":"2023-10-30T06:24:21.477449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:24.666463+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:24:24.666463+0000","last_peered":"2023-10-30T06:24:24.666463+0000","last_clean":"2023-10-30T06:24:24.666463+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:24:24.666463+0000","last_undegraded":"2023-10-30T06:24:24.666463+0000","last_fullsized":"2023-10-30T06:24:24.666463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411335,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6809999999999992}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.797000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0879999999999992}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.712999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.305}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.949000000000002}]}]},{"osd":6,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.271000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.106}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.898999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.315000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.916}]}]},{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7750000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.664000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.281000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.707000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.504000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.805}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.077999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.53700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7539999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.933}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.492999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.140999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.559999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":32.021000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.274999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.398999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.388999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.969000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.931000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1639999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.337}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.733000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.124000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.361999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.518000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":87.515000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6550000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.058}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8770000000000007}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.205}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.445}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.763999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:24:29.889 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:24:29.903 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:25.104 86020== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.007 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:25.209 86020== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.064 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:25.250 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.064 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:25.266 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.119 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:25.305 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.121 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:25.322 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.289 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:25.490 86020== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.365 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:25.551 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.386 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:25.588 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:24:30.930 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:30.931 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:24:30.943 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-30T06:24:29.259963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218148,"kb_used_data":1960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756572,"statfs":{"total":773094113280,"available":772870729728,"internally_reserved":0,"allocated":2007040,"data_stored":1458048,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.595335"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.490583+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:24:21.490583+0000","last_peered":"2023-10-30T06:24:21.490583+0000","last_clean":"2023-10-30T06:24:21.490583+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:24:21.490583+0000","last_undegraded":"2023-10-30T06:24:21.490583+0000","last_fullsized":"2023-10-30T06:24:21.490583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.343934+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:24:21.343934+0000","last_peered":"2023-10-30T06:24:21.343934+0000","last_clean":"2023-10-30T06:24:21.343934+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:24:21.343934+0000","last_undegraded":"2023-10-30T06:24:21.343934+0000","last_fullsized":"2023-10-30T06:24:21.343934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.482197+0000","last_change":"2023-10-30T06:24:20.087820+0000","last_active":"2023-10-30T06:24:21.482197+0000","last_peered":"2023-10-30T06:24:21.482197+0000","last_clean":"2023-10-30T06:24:21.482197+0000","last_became_active":"2023-10-30T06:24:20.060063+0000","last_became_peered":"2023-10-30T06:24:20.060063+0000","last_unstale":"2023-10-30T06:24:21.482197+0000","last_undegraded":"2023-10-30T06:24:21.482197+0000","last_fullsized":"2023-10-30T06:24:21.482197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:25.280973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.459454+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:24:21.459454+0000","last_peered":"2023-10-30T06:24:21.459454+0000","last_clean":"2023-10-30T06:24:21.459454+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:24:21.459454+0000","last_undegraded":"2023-10-30T06:24:21.459454+0000","last_fullsized":"2023-10-30T06:24:21.459454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.486866+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:24:21.486866+0000","last_peered":"2023-10-30T06:24:21.486866+0000","last_clean":"2023-10-30T06:24:21.486866+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:24:21.486866+0000","last_undegraded":"2023-10-30T06:24:21.486866+0000","last_fullsized":"2023-10-30T06:24:21.486866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.467833+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:24:21.467833+0000","last_peered":"2023-10-30T06:24:21.467833+0000","last_clean":"2023-10-30T06:24:21.467833+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:24:21.467833+0000","last_undegraded":"2023-10-30T06:24:21.467833+0000","last_fullsized":"2023-10-30T06:24:21.467833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.472914+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:24:21.472914+0000","last_peered":"2023-10-30T06:24:21.472914+0000","last_clean":"2023-10-30T06:24:21.472914+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:24:21.472914+0000","last_undegraded":"2023-10-30T06:24:21.472914+0000","last_fullsized":"2023-10-30T06:24:21.472914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.477449+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:24:21.477449+0000","last_peered":"2023-10-30T06:24:21.477449+0000","last_clean":"2023-10-30T06:24:21.477449+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:24:21.477449+0000","last_undegraded":"2023-10-30T06:24:21.477449+0000","last_fullsized":"2023-10-30T06:24:21.477449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":117,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:24.666463+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:24:24.666463+0000","last_peered":"2023-10-30T06:24:24.666463+0000","last_clean":"2023-10-30T06:24:24.666463+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:24:24.666463+0000","last_undegraded":"2023-10-30T06:24:24.666463+0000","last_fullsized":"2023-10-30T06:24:24.666463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411335,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6809999999999992}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.797000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0879999999999992}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.712999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.305}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.949000000000002}]}]},{"osd":6,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.271000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.106}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2959999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.898999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.75}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.315000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.916}]}]},{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7750000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.664000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.281000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.707000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.504000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.805}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.077999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":139.53700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.7539999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.933}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.492999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.140999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.559999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":32.021000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.274999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.398999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.388999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.969000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.931000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1639999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.6929999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.337}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.733000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.124000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.361999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":35.518000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":87.515000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6550000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.058}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.52}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8770000000000007}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8070000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.205}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.445}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.763999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:24:30.945 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T06:24:30.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-10-30T06:24:31.562 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 24 2023-10-30T06:24:31.562 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 09a24e68-832d-4fbc-9604-48e808368d81 2023-10-30T06:24:31.562 INFO:teuthology.orchestra.run.smithi079.stdout:created 2023-10-30T06:23:23.243533+0000 2023-10-30T06:24:31.562 INFO:teuthology.orchestra.run.smithi079.stdout:modified 2023-10-30T06:24:21.220169+0000 2023-10-30T06:24:31.563 INFO:teuthology.orchestra.run.smithi079.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-30T06:24:31.563 INFO:teuthology.orchestra.run.smithi079.stdout:crush_version 7 2023-10-30T06:24:31.563 INFO:teuthology.orchestra.run.smithi079.stdout:full_ratio 0.95 2023-10-30T06:24:31.563 INFO:teuthology.orchestra.run.smithi079.stdout:backfillfull_ratio 0.9 2023-10-30T06:24:31.564 INFO:teuthology.orchestra.run.smithi079.stdout:nearfull_ratio 0.85 2023-10-30T06:24:31.564 INFO:teuthology.orchestra.run.smithi079.stdout:require_min_compat_client luminous 2023-10-30T06:24:31.564 INFO:teuthology.orchestra.run.smithi079.stdout:min_compat_client jewel 2023-10-30T06:24:31.564 INFO:teuthology.orchestra.run.smithi079.stdout:require_osd_release reef 2023-10-30T06:24:31.564 INFO:teuthology.orchestra.run.smithi079.stdout:stretch_mode_enabled false 2023-10-30T06:24:31.565 INFO:teuthology.orchestra.run.smithi079.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2023-10-30T06:24:31.565 INFO:teuthology.orchestra.run.smithi079.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 24 flags hashpspool stripe_width 0 application rbd read_balance_score 4.98 2023-10-30T06:24:31.565 INFO:teuthology.orchestra.run.smithi079.stdout:max_osd 8 2023-10-30T06:24:31.565 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6810/2677821092,v1:172.21.15.79:6811/2677821092] [v2:172.21.15.79:6812/2677821092,v1:172.21.15.79:6813/2677821092] exists,up 6e79cc30-6aa7-489d-8806-48c107644b8b 2023-10-30T06:24:31.565 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1 up in weight 1 up_from 18 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6826/2076704984,v1:172.21.15.79:6827/2076704984] [v2:172.21.15.79:6828/2076704984,v1:172.21.15.79:6829/2076704984] exists,up c8a16891-3cdc-487f-9fdc-1b88ce43caf4 2023-10-30T06:24:31.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6818/3346893925,v1:172.21.15.79:6819/3346893925] [v2:172.21.15.79:6820/3346893925,v1:172.21.15.79:6821/3346893925] exists,up 0b1c9a57-cced-4163-a178-f461fc9596aa 2023-10-30T06:24:31.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3 up in weight 1 up_from 16 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6802/1649947840,v1:172.21.15.79:6803/1649947840] [v2:172.21.15.79:6804/1649947840,v1:172.21.15.79:6805/1649947840] exists,up eab98798-a4c8-4e5f-83ec-40b3f31a4d50 2023-10-30T06:24:31.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/2788056220,v1:172.21.15.112:6825/2788056220] [v2:172.21.15.112:6826/2788056220,v1:172.21.15.112:6827/2788056220] exists,up 6f17fdbb-5762-457a-8b89-4c5a4f2197b2 2023-10-30T06:24:31.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/2352822260,v1:172.21.15.112:6809/2352822260] [v2:172.21.15.112:6810/2352822260,v1:172.21.15.112:6811/2352822260] exists,up 49ea8aae-4b05-43c9-979a-99026ae58a89 2023-10-30T06:24:31.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/4229386639,v1:172.21.15.112:6801/4229386639] [v2:172.21.15.112:6802/4229386639,v1:172.21.15.112:6803/4229386639] exists,up ebb325d3-e1fc-44b6-929b-b7fdd71f82b1 2023-10-30T06:24:31.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/3711700983,v1:172.21.15.112:6817/3711700983] [v2:172.21.15.112:6818/3711700983,v1:172.21.15.112:6819/3711700983] exists,up 31813e1b-e740-49b2-9173-85911e32f3d9 2023-10-30T06:24:31.574 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6810/2677821092,v1:172.21.15.79:6811/2677821092] [v2:172.21.15.79:6812/2677821092,v1:172.21.15.79:6813/2677821092] exists,up 6e79cc30-6aa7-489d-8806-48c107644b8b', 'osd.1 up in weight 1 up_from 18 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6826/2076704984,v1:172.21.15.79:6827/2076704984] [v2:172.21.15.79:6828/2076704984,v1:172.21.15.79:6829/2076704984] exists,up c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'osd.2 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6818/3346893925,v1:172.21.15.79:6819/3346893925] [v2:172.21.15.79:6820/3346893925,v1:172.21.15.79:6821/3346893925] exists,up 0b1c9a57-cced-4163-a178-f461fc9596aa', 'osd.3 up in weight 1 up_from 16 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6802/1649947840,v1:172.21.15.79:6803/1649947840] [v2:172.21.15.79:6804/1649947840,v1:172.21.15.79:6805/1649947840] exists,up eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'osd.4 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6824/2788056220,v1:172.21.15.112:6825/2788056220] [v2:172.21.15.112:6826/2788056220,v1:172.21.15.112:6827/2788056220] exists,up 6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'osd.5 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6808/2352822260,v1:172.21.15.112:6809/2352822260] [v2:172.21.15.112:6810/2352822260,v1:172.21.15.112:6811/2352822260] exists,up 49ea8aae-4b05-43c9-979a-99026ae58a89', 'osd.6 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6800/4229386639,v1:172.21.15.112:6801/4229386639] [v2:172.21.15.112:6802/4229386639,v1:172.21.15.112:6803/4229386639] exists,up ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'osd.7 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6816/3711700983,v1:172.21.15.112:6817/3711700983] [v2:172.21.15.112:6818/3711700983,v1:172.21.15.112:6819/3711700983] exists,up 31813e1b-e740-49b2-9173-85911e32f3d9'] 2023-10-30T06:24:31.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-10-30T06:24:31.757 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "mon.a", 2023-10-30T06:24:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster": "ceph", 2023-10-30T06:24:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-10-30T06:24:31.758 INFO:teuthology.orchestra.run.smithi079.stdout: "admin_socket_mode": "", 2023-10-30T06:24:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-10-30T06:24:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_client_required": "cephx, none", 2023-10-30T06:24:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_cluster_required": "cephx", 2023-10-30T06:24:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_debug": "true", 2023-10-30T06:24:31.759 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_required": "cephx", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_service_ticket_ttl": "240.000000", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_supported": "cephx", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio": "true", 2023-10-30T06:24:31.760 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_max_queue_depth": "1024", 2023-10-30T06:24:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_poll_ms": "250", 2023-10-30T06:24:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_aio_reap_max": "16", 2023-10-30T06:24:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_async_discard": "false", 2023-10-30T06:24:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_block_size": "4096", 2023-10-30T06:24:31.761 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio": "false", 2023-10-30T06:24:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-10-30T06:24:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-10-30T06:24:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_debug_inflight_ios": "false", 2023-10-30T06:24:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_enable_discard": "false", 2023-10-30T06:24:31.762 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry": "3", 2023-10-30T06:24:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_flock_retry_interval": "0.100000", 2023-10-30T06:24:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash": "0", 2023-10-30T06:24:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_inject_crash_flush_delay": "2", 2023-10-30T06:24:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring": "false", 2023-10-30T06:24:31.763 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_hipri": "false", 2023-10-30T06:24:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_ioring_sqthread_poll": "false", 2023-10-30T06:24:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-10-30T06:24:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_buffer_alignment": "4096", 2023-10-30T06:24:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-10-30T06:24:31.764 INFO:teuthology.orchestra.run.smithi079.stdout: "bdev_type": "", 2023-10-30T06:24:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_alloc_size": "1048576", 2023-10-30T06:24:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_allocator": "hybrid", 2023-10-30T06:24:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_buffered_io": "true", 2023-10-30T06:24:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_for_zeros": "false", 2023-10-30T06:24:31.765 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_often": "false", 2023-10-30T06:24:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-10-30T06:24:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_compact_log_sync": "false", 2023-10-30T06:24:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-10-30T06:24:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-10-30T06:24:31.766 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_compact_min_size": "16777216", 2023-10-30T06:24:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_log_replay_check_allocations": "true", 2023-10-30T06:24:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_log_runway": "4194304", 2023-10-30T06:24:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_max_prefetch": "1048576", 2023-10-30T06:24:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_flush_size": "524288", 2023-10-30T06:24:31.767 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_min_log_runway": "1048576", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery": "false", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_shared_alloc_size": "65536", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_sync_write": "false", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-10-30T06:24:31.768 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-10-30T06:24:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-10-30T06:24:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocation_from_file": "true", 2023-10-30T06:24:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_allocator": "hybrid", 2023-10-30T06:24:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-10-30T06:24:31.769 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-10-30T06:24:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-10-30T06:24:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-10-30T06:24:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-10-30T06:24:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-10-30T06:24:31.770 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blobid_prealloc": "10240", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_create": "true", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_create": "false", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_path": "", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_db_size": "0", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_path": "", 2023-10-30T06:24:31.771 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_preallocate_file": "false", 2023-10-30T06:24:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_size": "107374182400", 2023-10-30T06:24:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_create": "false", 2023-10-30T06:24:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_path": "", 2023-10-30T06:24:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_block_wal_size": "100663296", 2023-10-30T06:24:31.772 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs": "true", 2023-10-30T06:24:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-10-30T06:24:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_env_mirror": "false", 2023-10-30T06:24:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-10-30T06:24:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-10-30T06:24:31.773 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-10-30T06:24:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-10-30T06:24:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-10-30T06:24:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-10-30T06:24:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune": "true", 2023-10-30T06:24:31.774 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-10-30T06:24:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-10-30T06:24:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-10-30T06:24:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-10-30T06:24:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size": "0", 2023-10-30T06:24:31.775 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-10-30T06:24:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-10-30T06:24:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-10-30T06:24:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-10-30T06:24:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_type": "2q", 2023-10-30T06:24:31.776 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-10-30T06:24:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_clone_cow": "true", 2023-10-30T06:24:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_algorithm": "snappy", 2023-10-30T06:24:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size": "0", 2023-10-30T06:24:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-10-30T06:24:31.777 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-10-30T06:24:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size": "0", 2023-10-30T06:24:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-10-30T06:24:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-10-30T06:24:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_mode": "none", 2023-10-30T06:24:31.778 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-10-30T06:24:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_csum_type": "crc32c", 2023-10-30T06:24:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_enforce_settings": "default", 2023-10-30T06:24:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_freelist": "false", 2023-10-30T06:24:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_fsck_abort": "false", 2023-10-30T06:24:31.779 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_inject_read_err": "false", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_legacy_omap": "false", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-10-30T06:24:31.780 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-10-30T06:24:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-10-30T06:24:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefill": "0.000000", 2023-10-30T06:24:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-10-30T06:24:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-10-30T06:24:31.781 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-10-30T06:24:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_small_allocations": "0", 2023-10-30T06:24:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-10-30T06:24:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_read": "true", 2023-10-30T06:24:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_default_buffered_write": "false", 2023-10-30T06:24:31.782 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops": "0", 2023-10-30T06:24:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-10-30T06:24:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-10-30T06:24:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_elastic_shared_blobs": "true", 2023-10-30T06:24:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-10-30T06:24:31.783 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-10-30T06:24:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-10-30T06:24:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-10-30T06:24:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-10-30T06:24:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fail_eio": "false", 2023-10-30T06:24:31.784 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-10-30T06:24:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-10-30T06:24:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-10-30T06:24:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-10-30T06:24:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs": "true", 2023-10-30T06:24:31.785 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-10-30T06:24:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount": "false", 2023-10-30T06:24:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-10-30T06:24:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount": "false", 2023-10-30T06:24:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-10-30T06:24:31.786 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-10-30T06:24:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-10-30T06:24:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-10-30T06:24:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-10-30T06:24:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-10-30T06:24:31.787 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_ignore_data_csum": "false", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_kvbackend": "rocksdb", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-10-30T06:24:31.788 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-10-30T06:24:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_log_op_age": "5.000000", 2023-10-30T06:24:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_alloc_size": "0", 2023-10-30T06:24:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size": "0", 2023-10-30T06:24:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-10-30T06:24:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-10-30T06:24:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_defer_interval": "3.000000", 2023-10-30T06:24:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_max_deferred_txc": "32", 2023-10-30T06:24:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size": "0", 2023-10-30T06:24:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-10-30T06:24:31.790 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_nid_prealloc": "1024", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size": "0", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_qfsck_on_mount": "true", 2023-10-30T06:24:31.791 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_retry_disk_reads": "3", 2023-10-30T06:24:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cf": "true", 2023-10-30T06:24:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2023-10-30T06:24:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2023-10-30T06:24:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_rocksdb_options_annex": "", 2023-10-30T06:24:31.792 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_coremask": "0x1", 2023-10-30T06:24:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_io_sleep": "5", 2023-10-30T06:24:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_max_io_completion": "0", 2023-10-30T06:24:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_spdk_mem": "512", 2023-10-30T06:24:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_sync_submit_transaction": "false", 2023-10-30T06:24:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_bytes": "67108864", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io": "0", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-10-30T06:24:31.794 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_tracing": "false", 2023-10-30T06:24:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-10-30T06:24:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-10-30T06:24:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved": "0", 2023-10-30T06:24:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-10-30T06:24:31.795 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-10-30T06:24:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-10-30T06:24:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-10-30T06:24:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-10-30T06:24:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-10-30T06:24:31.796 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_zero_block_detection": "false", 2023-10-30T06:24:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-10-30T06:24:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_action_update_interval": "2", 2023-10-30T06:24:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-10-30T06:24:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-10-30T06:24:31.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-10-30T06:24:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-10-30T06:24:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_mount_timeout": "10", 2023-10-30T06:24:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-10-30T06:24:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-10-30T06:24:31.798 INFO:teuthology.orchestra.run.smithi079.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-10-30T06:24:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-10-30T06:24:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-10-30T06:24:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-10-30T06:24:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_signatures": "false", 2023-10-30T06:24:31.799 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_cluster_require_version": "2", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_signatures": "false", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_require_version": "2", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_signatures": "false", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_service_require_version": "2", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "cephx_sign_messages": "true", 2023-10-30T06:24:31.800 INFO:teuthology.orchestra.run.smithi079.stdout: "chdir": "", 2023-10-30T06:24:31.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_acl_type": "", 2023-10-30T06:24:31.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_asio_thread_count": "2", 2023-10-30T06:24:31.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_mid": "0.750000", 2023-10-30T06:24:31.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_cache_size": "16384", 2023-10-30T06:24:31.801 INFO:teuthology.orchestra.run.smithi079.stdout: "client_caps_release_delay": "5", 2023-10-30T06:24:31.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_check_pool_perm": "true", 2023-10-30T06:24:31.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_collect_and_send_global_metrics": "false", 2023-10-30T06:24:31.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_force_sync_read": "false", 2023-10-30T06:24:31.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_getattr_caps": "false", 2023-10-30T06:24:31.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_debug_inject_tick_delay": "0", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_die_on_failed_remount": "false", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_dirsize_rbytes": "true", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_force_lazyio": "false", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_fs": "", 2023-10-30T06:24:31.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_fixed_oldest_tid": "false", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_inject_release_failure": "false", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_inline_size": "4096", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_max_retries_on_remount_failure": "5", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mds_namespace": "", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_metadata": "", 2023-10-30T06:24:31.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_gid": "-1", 2023-10-30T06:24:31.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_timeout": "300", 2023-10-30T06:24:31.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mount_uid": "-1", 2023-10-30T06:24:31.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_mountpoint": "/", 2023-10-30T06:24:31.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_notify_timeout": "10", 2023-10-30T06:24:31.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc": "true", 2023-10-30T06:24:31.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty": "104857600", 2023-10-30T06:24:31.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_dirty_age": "5.000000", 2023-10-30T06:24:31.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_max_objects": "1000", 2023-10-30T06:24:31.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_size": "209715200", 2023-10-30T06:24:31.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_oc_target_dirty": "8388608", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_permissions": "true", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota": "true", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_quota_df": "true", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_bytes": "0", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_max_periods": "4", 2023-10-30T06:24:31.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_readahead_min": "131072", 2023-10-30T06:24:31.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_reconnect_stale": "false", 2023-10-30T06:24:31.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_shutdown_timeout": "30", 2023-10-30T06:24:31.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_snapdir": ".snap", 2023-10-30T06:24:31.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_tick_interval": "1", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_trace": "", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_try_dentry_invalidate": "false", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_faked_inos": "false", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_use_random_mds": "false", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog": "false", 2023-10-30T06:24:31.809 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-10-30T06:24:31.810 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_graylog_port": "12201", 2023-10-30T06:24:31.810 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_monitors": "default=true", 2023-10-30T06:24:31.810 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog": "false", 2023-10-30T06:24:31.810 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-10-30T06:24:31.810 INFO:teuthology.orchestra.run.smithi079.stdout: "clog_to_syslog_level": "info", 2023-10-30T06:24:31.811 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "-", 2023-10-30T06:24:31.811 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network": "", 2023-10-30T06:24:31.811 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_network_interface": "", 2023-10-30T06:24:31.811 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_isal": "false", 2023-10-30T06:24:31.811 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_level": "5", 2023-10-30T06:24:31.812 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zlib_winsize": "-15", 2023-10-30T06:24:31.812 INFO:teuthology.orchestra.run.smithi079.stdout: "compressor_zstd_level": "1", 2023-10-30T06:24:31.812 INFO:teuthology.orchestra.run.smithi079.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-10-30T06:24:31.812 INFO:teuthology.orchestra.run.smithi079.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-10-30T06:24:31.812 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_op_num_threads": "6", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_alien_thread_cpu_cores": "", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_obc_lru_size": "10", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crimson_seastar_smp": "1", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "", 2023-10-30T06:24:31.813 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook": "", 2023-10-30T06:24:31.814 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location_hook_timeout": "10", 2023-10-30T06:24:31.814 INFO:teuthology.orchestra.run.smithi079.stdout: "daemonize": "false", 2023-10-30T06:24:31.814 INFO:teuthology.orchestra.run.smithi079.stdout: "daos_pool": "tank", 2023-10-30T06:24:31.814 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-10-30T06:24:31.814 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "dbstore_db_name_prefix": "dbstore", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_alienstore": "0/5", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_allow_any_pool_priority": "false", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok": "1/5", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asok_assert_abort": "false", 2023-10-30T06:24:31.815 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_asserts_on_shutdown": "true", 2023-10-30T06:24:31.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_auth": "20/20", 2023-10-30T06:24:31.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bdev": "1/3", 2023-10-30T06:24:31.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluefs": "1/5", 2023-10-30T06:24:31.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_bluestore": "1/5", 2023-10-30T06:24:31.816 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_buffer": "0/1", 2023-10-30T06:24:31.817 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ceph_exporter": "1/5", 2023-10-30T06:24:31.817 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephfs_mirror": "0/5", 2023-10-30T06:24:31.817 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cephsqlite": "0/5", 2023-10-30T06:24:31.817 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_client": "0/5", 2023-10-30T06:24:31.817 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_compressor": "1/5", 2023-10-30T06:24:31.818 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_context": "0/1", 2023-10-30T06:24:31.818 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crush": "1/1", 2023-10-30T06:24:31.818 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_crypto": "1/5", 2023-10-30T06:24:31.818 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_cyanstore": "0/5", 2023-10-30T06:24:31.818 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_deliberately_leak_memory": "false", 2023-10-30T06:24:31.819 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_disable_randomized_ping": "false", 2023-10-30T06:24:31.819 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_dpdk": "1/5", 2023-10-30T06:24:31.819 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_eventtrace": "1/5", 2023-10-30T06:24:31.819 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filer": "0/1", 2023-10-30T06:24:31.819 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_filestore": "1/3", 2023-10-30T06:24:31.820 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_finisher": "1/1", 2023-10-30T06:24:31.820 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_fuse": "1/5", 2023-10-30T06:24:31.820 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeat_testing_span": "0", 2023-10-30T06:24:31.820 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_heartbeatmap": "1/5", 2023-10-30T06:24:31.820 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_immutable_obj_cache": "0/5", 2023-10-30T06:24:31.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_javaclient": "1/5", 2023-10-30T06:24:31.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journal": "1/3", 2023-10-30T06:24:31.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_journaler": "0/5", 2023-10-30T06:24:31.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_kstore": "1/5", 2023-10-30T06:24:31.821 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_lockdep": "0/1", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mclock": "1/5", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds": "1/5", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_balancer": "1/5", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_locker": "1/5", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log": "1/5", 2023-10-30T06:24:31.822 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_log_expire": "1/5", 2023-10-30T06:24:31.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mds_migrator": "1/5", 2023-10-30T06:24:31.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_memstore": "1/5", 2023-10-30T06:24:31.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgr": "2/5", 2023-10-30T06:24:31.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mgrc": "1/5", 2023-10-30T06:24:31.823 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_mon": "20/20", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_monc": "0/10", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_ms": "1/1", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_none": "0/5", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objclass": "0/5", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objectcacher": "0/5", 2023-10-30T06:24:31.824 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_objecter": "0/1", 2023-10-30T06:24:31.825 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_optracker": "0/5", 2023-10-30T06:24:31.825 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_osd": "1/5", 2023-10-30T06:24:31.825 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_paxos": "20/20", 2023-10-30T06:24:31.825 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_perfcounter": "1/5", 2023-10-30T06:24:31.825 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_prioritycache": "1/5", 2023-10-30T06:24:31.826 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rados": "0/5", 2023-10-30T06:24:31.826 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd": "0/5", 2023-10-30T06:24:31.826 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_mirror": "0/5", 2023-10-30T06:24:31.826 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_pwl": "0/5", 2023-10-30T06:24:31.826 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rbd_replay": "0/5", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_refs": "0/0", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_reserver": "1/1", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw": "1/5", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_access": "1/5", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_datacache": "1/5", 2023-10-30T06:24:31.827 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_dbstore": "1/5", 2023-10-30T06:24:31.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_flight": "1/5", 2023-10-30T06:24:31.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rgw_sync": "1/5", 2023-10-30T06:24:31.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_rocksdb": "4/5", 2023-10-30T06:24:31.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore": "0/5", 2023-10-30T06:24:31.828 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_backref": "0/5", 2023-10-30T06:24:31.829 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cache": "0/5", 2023-10-30T06:24:31.829 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_cleaner": "0/5", 2023-10-30T06:24:31.829 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_device": "0/5", 2023-10-30T06:24:31.829 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_epm": "0/5", 2023-10-30T06:24:31.829 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-10-30T06:24:31.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_journal": "0/5", 2023-10-30T06:24:31.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_lba": "0/5", 2023-10-30T06:24:31.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_odata": "0/5", 2023-10-30T06:24:31.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_omap": "0/5", 2023-10-30T06:24:31.830 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_onode": "0/5", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_t": "0/5", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_seastore_tm": "0/5", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_striper": "0/1", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_test": "0/5", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_throttle": "1/1", 2023-10-30T06:24:31.831 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_timer": "0/1", 2023-10-30T06:24:31.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_tp": "0/5", 2023-10-30T06:24:31.832 INFO:teuthology.orchestra.run.smithi079.stdout: "debug_trace": "1/5", 2023-10-30T06:24:31.832 INFO:teuthology.orchestra.run.smithi079.stdout: "defer_client_eviction_on_laggy_osds": "true", 2023-10-30T06:24:31.832 INFO:teuthology.orchestra.run.smithi079.stdout: "device_failure_prediction_mode": "none", 2023-10-30T06:24:31.832 INFO:teuthology.orchestra.run.smithi079.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-10-30T06:24:31.833 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-10-30T06:24:31.833 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_graylog": "false", 2023-10-30T06:24:31.833 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_journald": "false", 2023-10-30T06:24:31.833 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_stderr": "true", 2023-10-30T06:24:31.833 INFO:teuthology.orchestra.run.smithi079.stdout: "err_to_syslog": "false", 2023-10-30T06:24:31.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event_tracing": "false", 2023-10-30T06:24:31.834 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_addr": "0.0.0.0", 2023-10-30T06:24:31.834 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_http_port": "9926", 2023-10-30T06:24:31.834 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_prio_limit": "5", 2023-10-30T06:24:31.835 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-10-30T06:24:31.835 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_sort_metrics": "true", 2023-10-30T06:24:31.835 INFO:teuthology.orchestra.run.smithi079.stdout: "exporter_stats_period": "5", 2023-10-30T06:24:31.835 INFO:teuthology.orchestra.run.smithi079.stdout: "fake_statfs_for_testing": "0", 2023-10-30T06:24:31.835 INFO:teuthology.orchestra.run.smithi079.stdout: "fatal_signal_handlers": "true", 2023-10-30T06:24:31.836 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_purge_ops": "10", 2023-10-30T06:24:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "filer_max_truncate_ops": "128", 2023-10-30T06:24:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_apply_finisher_threads": "1", 2023-10-30T06:24:31.837 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_blackhole": "false", 2023-10-30T06:24:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_clone_range": "true", 2023-10-30T06:24:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_btrfs_snap": "true", 2023-10-30T06:24:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_caller_concurrency": "10", 2023-10-30T06:24:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_collect_device_partition_information": "true", 2023-10-30T06:24:31.838 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_commit_timeout": "600.000000", 2023-10-30T06:24:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_inject_read_err": "false", 2023-10-30T06:24:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_omap_check": "false", 2023-10-30T06:24:31.839 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_random_read_err": "0.000000", 2023-10-30T06:24:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_debug_verify_split": "false", 2023-10-30T06:24:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_dump_file": "", 2023-10-30T06:24:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-10-30T06:24:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-10-30T06:24:31.840 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fadvise": "true", 2023-10-30T06:24:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fail_eio": "true", 2023-10-30T06:24:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_shards": "16", 2023-10-30T06:24:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fd_cache_size": "128", 2023-10-30T06:24:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap": "false", 2023-10-30T06:24:31.841 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fiemap_threshold": "4096", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_index_retry_probability": "0.000000", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_inject_stall": "0", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_parallel": "false", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_trailing": "false", 2023-10-30T06:24:31.842 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_journal_writeahead": "false", 2023-10-30T06:24:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_kill_at": "0", 2023-10-30T06:24:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-10-30T06:24:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size": "0", 2023-10-30T06:24:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-10-30T06:24:31.843 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-10-30T06:24:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-10-30T06:24:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs": "0", 2023-10-30T06:24:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-10-30T06:24:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_other": "2", 2023-10-30T06:24:31.844 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-10-30T06:24:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_sync_interval": "5.000000", 2023-10-30T06:24:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size": "0", 2023-10-30T06:24:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-10-30T06:24:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-10-30T06:24:31.845 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-10-30T06:24:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_merge_threshold": "-10", 2023-10-30T06:24:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_min_sync_interval": "0.010000", 2023-10-30T06:24:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_odsync_write": "false", 2023-10-30T06:24:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend": "rocksdb", 2023-10-30T06:24:31.846 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_backend_path": "", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_omap_header_cache_size": "1024", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_ondisk_finisher_threads": "1", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_thread_timeout": "60", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_op_threads": "2", 2023-10-30T06:24:31.847 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_punch_hole": "false", 2023-10-30T06:24:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-10-30T06:24:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-10-30T06:24:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-10-30T06:24:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-10-30T06:24:31.848 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-10-30T06:24:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_bytes": "104857600", 2023-10-30T06:24:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-10-30T06:24:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-10-30T06:24:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-10-30T06:24:31.849 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_queue_max_ops": "50", 2023-10-30T06:24:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-10-30T06:24:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_seek_data_hole": "false", 2023-10-30T06:24:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc": "false", 2023-10-30T06:24:31.850 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-10-30T06:24:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_splice": "false", 2023-10-30T06:24:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_multiple": "2", 2023-10-30T06:24:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_split_rand_factor": "20", 2023-10-30T06:24:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_update_to": "1000", 2023-10-30T06:24:31.851 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-10-30T06:24:31.852 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_enable": "true", 2023-10-30T06:24:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-10-30T06:24:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-10-30T06:24:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-10-30T06:24:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-10-30T06:24:31.853 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-10-30T06:24:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-10-30T06:24:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_xfs_extsize": "false", 2023-10-30T06:24:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "filestore_zfs_snap": "false", 2023-10-30T06:24:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "fio_dir": "/tmp/fio", 2023-10-30T06:24:31.854 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_allow_other": "true", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_atomic_o_trunc": "true", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_big_writes": "true", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_debug": "false", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_default_permissions": "false", 2023-10-30T06:24:31.855 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_disable_pagecache": "false", 2023-10-30T06:24:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_max_write": "0", 2023-10-30T06:24:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_multithreaded": "true", 2023-10-30T06:24:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_require_active_mds": "true", 2023-10-30T06:24:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_set_user_groups": "true", 2023-10-30T06:24:31.856 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_move": "true", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_read": "true", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_splice_write": "true", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_syncfs_on_mksnap": "true", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "fuse_use_invalidate_cb": "true", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-10-30T06:24:31.857 INFO:teuthology.orchestra.run.smithi079.stdout: "gss_target_name": "ceph", 2023-10-30T06:24:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_file": "", 2023-10-30T06:24:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_inject_failure": "0", 2023-10-30T06:24:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_interval": "5", 2023-10-30T06:24:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "host": "smithi079", 2023-10-30T06:24:31.858 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-10-30T06:24:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-10-30T06:24:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_max_size": "1073741824", 2023-10-30T06:24:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-10-30T06:24:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-10-30T06:24:31.859 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-10-30T06:24:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-10-30T06:24:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-10-30T06:24:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-10-30T06:24:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-10-30T06:24:31.860 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "immutable_object_cache_watermark": "0.900000", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "inject_early_sigterm": "false", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_agent_port": "6799", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "jaeger_tracing_enable": "false", 2023-10-30T06:24:31.861 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_aio": "true", 2023-10-30T06:24:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_align_min_size": "65536", 2023-10-30T06:24:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_align": "true", 2023-10-30T06:24:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_block_size": "4096", 2023-10-30T06:24:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_dio": "true", 2023-10-30T06:24:31.862 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_discard": "false", 2023-10-30T06:24:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_force_aio": "false", 2023-10-30T06:24:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_ignore_corruption": "false", 2023-10-30T06:24:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_bytes": "10485760", 2023-10-30T06:24:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_max_write_entries": "100", 2023-10-30T06:24:31.863 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_replay_from": "0", 2023-10-30T06:24:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_multiple": "0.000000", 2023-10-30T06:24:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-10-30T06:24:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-10-30T06:24:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_throttle_max_multiple": "0.000000", 2023-10-30T06:24:31.864 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_write_header_frequency": "0", 2023-10-30T06:24:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "journal_zero_on_create": "false", 2023-10-30T06:24:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prefetch_periods": "10", 2023-10-30T06:24:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_prezero_periods": "5", 2023-10-30T06:24:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "journaler_write_head_interval": "15", 2023-10-30T06:24:31.865 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "", 2023-10-30T06:24:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "keyfile": "", 2023-10-30T06:24:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-10-30T06:24:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_backend": "rocksdb", 2023-10-30T06:24:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_default_stripe_size": "65536", 2023-10-30T06:24:31.866 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount": "false", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_fsck_on_mount_deep": "true", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_bytes": "67108864", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_max_ops": "512", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_nid_prealloc": "1024", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_onode_map_size": "1024", 2023-10-30T06:24:31.867 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-10-30T06:24:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_submit_transaction": "false", 2023-10-30T06:24:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "kstore_sync_transaction": "false", 2023-10-30T06:24:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "librados_thread_count": "2", 2023-10-30T06:24:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep": "false", 2023-10-30T06:24:31.868 INFO:teuthology.orchestra.run.smithi079.stdout: "lockdep_force_backtrace": "false", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_coarse_timestamps": "true", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_flush_on_exit": "false", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_host": "127.0.0.1", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_graylog_port": "12201", 2023-10-30T06:24:31.869 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_new": "1000", 2023-10-30T06:24:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_max_recent": "10000", 2023-10-30T06:24:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stderr_prefix": "", 2023-10-30T06:24:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_stop_at_utilization": "0.970000", 2023-10-30T06:24:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_file": "true", 2023-10-30T06:24:31.870 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_graylog": "false", 2023-10-30T06:24:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_journald": "false", 2023-10-30T06:24:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_stderr": "false", 2023-10-30T06:24:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "log_to_syslog": "false", 2023-10-30T06:24:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "max_rotating_auth_attempts": "10", 2023-10-30T06:24:31.871 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2023-10-30T06:24:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_action_on_write_error": "1", 2023-10-30T06:24:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_alternate_name_max": "8192", 2023-10-30T06:24:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_asio_thread_count": "2", 2023-10-30T06:24:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_export_pin": "true", 2023-10-30T06:24:31.872 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_dirs": "true", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_interval": "5", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_fragment_size_max": "100000", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_idle_threshold": "0.000000", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_interval": "10", 2023-10-30T06:24:31.873 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max": "-1", 2023-10-30T06:24:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_max_until": "-1", 2023-10-30T06:24:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_merge_size": "50", 2023-10-30T06:24:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_midchunk": "0.300000", 2023-10-30T06:24:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_rebalance": "0.100000", 2023-10-30T06:24:31.874 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_min_start": "0.200000", 2023-10-30T06:24:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_minchunk": "0.001000", 2023-10-30T06:24:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_mode": "0", 2023-10-30T06:24:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_max": "1.200000", 2023-10-30T06:24:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_need_min": "0.800000", 2023-10-30T06:24:31.875 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_overload_epochs": "2", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_sample_interval": "3.000000", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_bits": "3", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_rd": "25000.000000", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_size": "10000", 2023-10-30T06:24:31.876 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_split_wr": "10000.000000", 2023-10-30T06:24:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_target_decay": "10.000000", 2023-10-30T06:24:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-10-30T06:24:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_grace": "15.000000", 2023-10-30T06:24:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_interval": "4.000000", 2023-10-30T06:24:31.877 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_beacon_mon_down_grace": "60", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_memory_limit": "4294967296", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_mid": "0.700000", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_release_free_interval": "10", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_reservation": "0.050000", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-10-30T06:24:31.878 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_interval": "1", 2023-10-30T06:24:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache_trim_threshold": "262144", 2023-10-30T06:24:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-10-30T06:24:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-10-30T06:24:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_delegate_inos_pct": "50", 2023-10-30T06:24:31.879 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_prealloc_inos": "1000", 2023-10-30T06:24:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-10-30T06:24:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_connect_bootstrapping": "false", 2023-10-30T06:24:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_damage_table_max_entries": "10000", 2023-10-30T06:24:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-10-30T06:24:31.880 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_auth_pins": "false", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_frag": "false", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_scatterstat": "false", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_debug_subtrees": "false", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_decay_halflife": "5.000000", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_default_dir_hash": "2", 2023-10-30T06:24:31.881 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_defer_session_stale": "true", 2023-10-30T06:24:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_deny_all_reconnect": "false", 2023-10-30T06:24:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_keys_per_op": "16384", 2023-10-30T06:24:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_commit_size": "10", 2023-10-30T06:24:31.882 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_max_entries": "0", 2023-10-30T06:24:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dir_prefetch": "true", 2023-10-30T06:24:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dirstat_min_interval": "1.000000", 2023-10-30T06:24:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_after_rejoin": "false", 2023-10-30T06:24:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_on_map": "false", 2023-10-30T06:24:31.883 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_file": "0", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_early_reply": "true", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enable_op_tracker": "true", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_enforce_unique_name": "true", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed": "true", 2023-10-30T06:24:31.884 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-10-30T06:24:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random": "true", 2023-10-30T06:24:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-10-30T06:24:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-10-30T06:24:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_forward_all_requests_to_auth": "false", 2023-10-30T06:24:31.885 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_go_bad_corrupt_dentry": "true", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_cache_threshold": "1.500000", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_health_summarize_threshold": "10", 2023-10-30T06:24:31.886 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_grace": "15.000000", 2023-10-30T06:24:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_heartbeat_reset_grace": "1000", 2023-10-30T06:24:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_health_dummy": "false", 2023-10-30T06:24:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2023-10-30T06:24:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_migrator_session_race": "false", 2023-10-30T06:24:31.887 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_skip_replaying_inotable": "false", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_join_fs": "", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_journal_format": "1", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_create_at": "0", 2023-10-30T06:24:31.888 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_export_at": "0", 2023-10-30T06:24:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_import_at": "0", 2023-10-30T06:24:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_at": "0", 2023-10-30T06:24:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_expire_at": "0", 2023-10-30T06:24:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_journal_replay_at": "0", 2023-10-30T06:24:31.889 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_link_at": "0", 2023-10-30T06:24:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_mdstable_at": "0", 2023-10-30T06:24:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_openc_at": "0", 2023-10-30T06:24:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_rename_at": "0", 2023-10-30T06:24:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_shutdown_at": "0", 2023-10-30T06:24:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_kill_skip_replaying_inotable": "false", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_event_large_threshold": "524288", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_events_per_segment": "1024", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_major_segment_event_ratio": "12", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_events": "-1", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_max_segments": "128", 2023-10-30T06:24:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_pause": "false", 2023-10-30T06:24:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_segment_size": "0", 2023-10-30T06:24:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_corrupt_events": "false", 2023-10-30T06:24:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_skip_unbounded_events": "false", 2023-10-30T06:24:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log_warn_factor": "2.000000", 2023-10-30T06:24:31.892 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_caps_per_client": "1048576", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_flushes": "100000", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_completed_requests": "100000", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_export_size": "20971520", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_file_recover": "32", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_files": "64", 2023-10-30T06:24:31.893 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops": "8192", 2023-10-30T06:24:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-10-30T06:24:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-10-30T06:24:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_max_snaps_per_dir": "100", 2023-10-30T06:24:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_metrics_update_interval": "2", 2023-10-30T06:24:31.894 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_per_client": "100", 2023-10-30T06:24:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_min_caps_working_set": "10000", 2023-10-30T06:24:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-10-30T06:24:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_numa_node": "-1", 2023-10-30T06:24:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-10-30T06:24:31.895 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_complaint_time": "30.000000", 2023-10-30T06:24:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_duration": "600", 2023-10-30T06:24:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_size": "20", 2023-10-30T06:24:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_size": "20", 2023-10-30T06:24:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_history_slow_op_threshold": "10", 2023-10-30T06:24:31.896 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_op_log_threshold": "5", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_grace": "15", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_ping_interval": "5", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_caps": "30000", 2023-10-30T06:24:31.897 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-10-30T06:24:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_max_decay_threshold": "131072", 2023-10-30T06:24:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-10-30T06:24:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_recall_warning_threshold": "262144", 2023-10-30T06:24:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_reconnect_timeout": "45.000000", 2023-10-30T06:24:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_interval": "1.000000", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_gid": "0", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_root_ino_uid": "0", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-10-30T06:24:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_evict": "true", 2023-10-30T06:24:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_blocklist_on_timeout": "true", 2023-10-30T06:24:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-10-30T06:24:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-10-30T06:24:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2023-10-30T06:24:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_cap_acquisition_throttle": "100000", 2023-10-30T06:24:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-10-30T06:24:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_session_metadata_threshold": "16777216", 2023-10-30T06:24:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-10-30T06:24:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_shutdown_check": "0", 2023-10-30T06:24:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_skip_ino": "0", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sleep_rank_change": "0.000000", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_max_uid": "4294967294", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_min_uid": "0", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_snap_rstat": "false", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_standby_replay_damaged": "false", 2023-10-30T06:24:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_symlink_recovery": "true", 2023-10-30T06:24:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_task_status_update_interval": "2.000000", 2023-10-30T06:24:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_exports": "0", 2023-10-30T06:24:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_thrash_fragments": "0", 2023-10-30T06:24:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_tick_interval": "5.000000", 2023-10-30T06:24:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_valgrind_exit": "false", 2023-10-30T06:24:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_backtrace": "1", 2023-10-30T06:24:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_verify_scatter": "false", 2023-10-30T06:24:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_ino_prealloc": "false", 2023-10-30T06:24:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_wipe_sessions": "false", 2023-10-30T06:24:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool_debug": "false", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_debug_omit_block_device_write": "false", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_device_bytes": "1073741824", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_set": "false", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "memstore_page_size": "65536", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_bytes": "134217728", 2023-10-30T06:24:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_messages": "512", 2023-10-30T06:24:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-10-30T06:24:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_connect_retry_interval": "1.000000", 2023-10-30T06:24:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-10-30T06:24:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-10-30T06:24:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_disabled_modules": "", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_max_pg_num_change": "128", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_bytes": "134217728", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mds_messages": "128", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-10-30T06:24:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_bytes": "134217728", 2023-10-30T06:24:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_mon_messages": "128", 2023-10-30T06:24:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_bytes": "536870912", 2023-10-30T06:24:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_osd_messages": "8192", 2023-10-30T06:24:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_pool": "true", 2023-10-30T06:24:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_service_beacon_grace": "60.000000", 2023-10-30T06:24:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_standby_modules": "true", 2023-10-30T06:24:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_period": "5", 2023-10-30T06:24:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_stats_threshold": "5", 2023-10-30T06:24:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_tick_period": "2", 2023-10-30T06:24:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-10-30T06:24:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_accept_timeout_factor": "2.000000", 2023-10-30T06:24:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_delete": "true", 2023-10-30T06:24:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_allow_pool_size_one": "true", 2023-10-30T06:24:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_auth_validate_all_caps": "true", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_bytes": "104857600", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_directed_command_retry": "5", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval": "3.000000", 2023-10-30T06:24:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-10-30T06:24:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-10-30T06:24:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-10-30T06:24:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_hunt_parallel": "3", 2023-10-30T06:24:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_log_interval": "1.000000", 2023-10-30T06:24:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-10-30T06:24:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_interval": "10.000000", 2023-10-30T06:24:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_ping_timeout": "30.000000", 2023-10-30T06:24:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_client_target_rank": "-1", 2023-10-30T06:24:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_allowed": "1.000000", 2023-10-30T06:24:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_file_level": "debug", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_file": "true", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog": "false", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-10-30T06:24:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-10-30T06:24:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_journald": "false", 2023-10-30T06:24:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_stderr": "false", 2023-10-30T06:24:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-10-30T06:24:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-10-30T06:24:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_bootstrap": "false", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_start": "false", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_compact_on_trim": "true", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_persist_interval": "10", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_con_tracker_score_halflife": "43200", 2023-10-30T06:24:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_config_key_max_entry_size": "65536", 2023-10-30T06:24:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_cpu_threads": "4", 2023-10-30T06:24:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_crush_min_required_version": "hammer", 2023-10-30T06:24:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_daemon_bytes": "419430400", 2023-10-30T06:24:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-10-30T06:24:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_crit": "5", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_avail_warn": "5", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_data_size_warn": "16106127360", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_block_osdmap_trim": "false", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_json": "false", 2023-10-30T06:24:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-10-30T06:24:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_dump_transactions": "false", 2023-10-30T06:24:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_extra_checks": "false", 2023-10-30T06:24:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-10-30T06:24:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-10-30T06:24:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_quincy": "false", 2023-10-30T06:24:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_no_require_reef": "false", 2023-10-30T06:24:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-10-30T06:24:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_delta_reset_interval": "10.000000", 2023-10-30T06:24:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_dns_srv_name": "ceph-mon", 2023-10-30T06:24:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_down_mkfs_grace": "120", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_default_strategy": "3", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_election_timeout": "5.000000", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_divisor": "2", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_elector_ping_timeout": "2.000000", 2023-10-30T06:24:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_enable_op_tracker": "true", 2023-10-30T06:24:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_fake_pool_delete": "false", 2023-10-30T06:24:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_force_quorum_join": "false", 2023-10-30T06:24:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_globalid_prealloc": "10000", 2023-10-30T06:24:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_detail_to_clog": "false", 2023-10-30T06:24:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_log_update_period": "5", 2023-10-30T06:24:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_max_detail": "50", 2023-10-30T06:24:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog": "true", 2023-10-30T06:24:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_interval": "600", 2023-10-30T06:24:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-10-30T06:24:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host": "172.21.15.79,[v2:172.21.15.79:3301,v1:172.21.15.79:6790],172.21.15.112", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_host_override": "", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_initial_members": "", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-10-30T06:24:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-10-30T06:24:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_keyvaluedb": "rocksdb", 2023-10-30T06:24:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease": "5.000000", 2023-10-30T06:24:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-10-30T06:24:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-10-30T06:24:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_full_interval": "50", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max": "10000", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_log_max_summary": "50", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_entries_per_event": "4096", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_log_epochs": "500", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mdsmap_epochs": "500", 2023-10-30T06:24:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_mgrmap_epochs": "500", 2023-10-30T06:24:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_osd": "10000", 2023-10-30T06:24:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pg_per_osd": "10000", 2023-10-30T06:24:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_pool_pg_num": "65536", 2023-10-30T06:24:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-10-30T06:24:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_force_trim_to": "0", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mds_skip_sanity": "false", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_autotune": "true", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_memory_target": "2147483648", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_beacon_grace": "30", 2023-10-30T06:24:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-10-30T06:24:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_digest_period": "5", 2023-10-30T06:24:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_inactive_grace": "60", 2023-10-30T06:24:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_mkfs_grace": "240", 2023-10-30T06:24:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-10-30T06:24:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_min_osdmap_epochs": "50", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_complaint_time": "30", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_duration": "600", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_size": "20", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_size": "20", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_history_slow_op_threshold": "10", 2023-10-30T06:24:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_op_log_threshold": "5", 2023-10-30T06:24:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-10-30T06:24:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-10-30T06:24:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-10-30T06:24:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_in": "false", 2023-10-30T06:24:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_auto_mark_new_in": "true", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size": "500", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_cache_size_min": "134217728", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_crush_smoke_test": "false", 2023-10-30T06:24:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_destroyed_out_interval": "600", 2023-10-30T06:24:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_interval": "600", 2023-10-30T06:24:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-10-30T06:24:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-10-30T06:24:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_force_trim_to": "0", 2023-10-30T06:24:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_full_ratio": "0.950000", 2023-10-30T06:24:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-10-30T06:24:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_halflife": "3600", 2023-10-30T06:24:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_max_interval": "300", 2023-10-30T06:24:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_laggy_weight": "0.300000", 2023-10-30T06:24:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-10-30T06:24:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_creating_pgs": "1024", 2023-10-30T06:24:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_max_initial_pgs": "1024", 2023-10-30T06:24:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_down_reporters": "2", 2023-10-30T06:24:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-10-30T06:24:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-10-30T06:24:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-10-30T06:24:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp": "true", 2023-10-30T06:24:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-10-30T06:24:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-10-30T06:24:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_report_timeout": "900", 2023-10-30T06:24:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-10-30T06:24:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-10-30T06:24:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_num_repaired": "10", 2023-10-30T06:24:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osd_warn_op_age": "32.000000", 2023-10-30T06:24:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-10-30T06:24:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_interval": "2", 2023-10-30T06:24:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_min": "15", 2023-10-30T06:24:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-10-30T06:24:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-10-30T06:24:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_stuck_threshold": "60", 2023-10-30T06:24:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-10-30T06:24:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_objects": "10000", 2023-10-30T06:24:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_per_osd": "0", 2023-10-30T06:24:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-10-30T06:24:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_crit_threshold": "0", 2023-10-30T06:24:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_pool_quota_warn_threshold": "0", 2023-10-30T06:24:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_probe_timeout": "2.000000", 2023-10-30T06:24:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_change": "0.050000", 2023-10-30T06:24:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_max_osds": "4", 2023-10-30T06:24:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-10-30T06:24:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-10-30T06:24:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-10-30T06:24:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-10-30T06:24:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_interval": "300", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_max_keys": "100", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_scrub_timeout": "300", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_session_timeout": "300", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_smart_report_timeout": "5", 2023-10-30T06:24:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stat_smooth_intervals": "6", 2023-10-30T06:24:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-10-30T06:24:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_min_size": "2", 2023-10-30T06:24:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_pool_size": "4", 2023-10-30T06:24:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-10-30T06:24:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_subscribe_interval": "86400.000000", 2023-10-30T06:24:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_debug": "false", 2023-10-30T06:24:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_keys": "2000", 2023-10-30T06:24:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_max_payload_size": "1048576", 2023-10-30T06:24:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_provider_kill_at": "0", 2023-10-30T06:24:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_requester_kill_at": "0", 2023-10-30T06:24:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_sync_timeout": "60.000000", 2023-10-30T06:24:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_target_pg_per_osd": "100", 2023-10-30T06:24:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_tick_interval": "5", 2023-10-30T06:24:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_interval": "300.000000", 2023-10-30T06:24:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-10-30T06:24:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_use_min_delay_socket": "false", 2023-10-30T06:24:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_older_version_delay": "604800", 2023-10-30T06:24:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-10-30T06:24:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-10-30T06:24:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-10-30T06:24:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_filestore_osds": "false", 2023-10-30T06:24:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-10-30T06:24:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-10-30T06:24:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-10-30T06:24:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_misplaced": "false", 2023-10-30T06:24:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-10-30T06:24:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_older_version": "true", 2023-10-30T06:24:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-10-30T06:24:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_app": "true", 2023-10-30T06:24:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-10-30T06:24:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-10-30T06:24:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-10-30T06:24:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-10-30T06:24:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_on_too_few_osds": "false", 2023-10-30T06:24:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-10-30T06:24:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-10-30T06:24:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": "", 2023-10-30T06:24:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-10-30T06:24:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2023-10-30T06:24:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-10-30T06:24:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-10-30T06:24:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-10-30T06:24:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-10-30T06:24:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "motr_tracing_enabled": "false", 2023-10-30T06:24:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_op_threads": "3", 2023-10-30T06:24:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_buffer_size": "131072", 2023-10-30T06:24:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_cm": "false", 2023-10-30T06:24:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_device_name": "", 2023-10-30T06:24:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_dscp": "96", 2023-10-30T06:24:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-10-30T06:24:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_gid_idx": "0", 2023-10-30T06:24:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_local_gid": "", 2023-10-30T06:24:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_polling_us": "1000", 2023-10-30T06:24:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_port_num": "1", 2023-10-30T06:24:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-10-30T06:24:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-10-30T06:24:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_roce_ver": "1", 2023-10-30T06:24:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_send_buffers": "1024", 2023-10-30T06:24:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_sl": "3", 2023-10-30T06:24:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_support_srq": "true", 2023-10-30T06:24:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_rdma_type": "ib", 2023-10-30T06:24:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_async_reap_threshold": "5", 2023-10-30T06:24:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_before_connect": "false", 2023-10-30T06:24:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv4": "true", 2023-10-30T06:24:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_ipv6": "false", 2023-10-30T06:24:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr1": "true", 2023-10-30T06:24:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_msgr2": "true", 2023-10-30T06:24:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_max": "7568", 2023-10-30T06:24:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_port_min": "6800", 2023-10-30T06:24:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_prefer_ipv4": "false", 2023-10-30T06:24:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_count": "3", 2023-10-30T06:24:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_bind_retry_delay": "5", 2023-10-30T06:24:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_client": "false", 2023-10-30T06:24:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mds": "false", 2023-10-30T06:24:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mgr": "false", 2023-10-30T06:24:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_mon": "false", 2023-10-30T06:24:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_blackhole_osd": "false", 2023-10-30T06:24:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_client_mode": "crc secure", 2023-10-30T06:24:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_mode": "crc secure", 2023-10-30T06:24:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_cluster_type": "", 2023-10-30T06:24:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_compress_secure": "false", 2023-10-30T06:24:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_idle_timeout": "900", 2023-10-30T06:24:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_connection_ready_timeout": "10", 2023-10-30T06:24:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_data": "true", 2023-10-30T06:24:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_crc_header": "true", 2023-10-30T06:24:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bad_msg": "false", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_bug": "true", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_old_message": "true", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_skipped_message": "false", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_die_on_unhandled_msg": "false", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-10-30T06:24:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_coremask": "0xF", 2023-10-30T06:24:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-10-30T06:24:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_devs_allowlist": "", 2023-10-30T06:24:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_enable_tso": "true", 2023-10-30T06:24:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-10-30T06:24:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-10-30T06:24:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hugepages": "", 2023-10-30T06:24:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_flow_control": "true", 2023-10-30T06:24:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-10-30T06:24:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_lro": "true", 2023-10-30T06:24:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_memory_channel": "4", 2023-10-30T06:24:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-10-30T06:24:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_pmd": "", 2023-10-30T06:24:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_port_id": "0", 2023-10-30T06:24:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-10-30T06:24:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_corrupt_message_level": "1", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_dump_on_send": "false", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_initial_backoff": "0.200000", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_max": "1.000000", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_probability": "0.000000", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_delay_type": "", 2023-10-30T06:24:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_internal_delays": "0.000000", 2023-10-30T06:24:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_network_congestion": "0", 2023-10-30T06:24:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_inject_socket_failures": "5000", 2023-10-30T06:24:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_learn_addr_from_peer": "true", 2023-10-30T06:24:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_accept_failures": "4", 2023-10-30T06:24:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_max_backoff": "15.000000", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_client_mode": "secure crc", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_cluster_mode": "secure crc", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_mon_service_mode": "secure crc", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_min_size": "1024", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compress_mode": "none", 2023-10-30T06:24:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_osd_compression_algorithm": "snappy", 2023-10-30T06:24:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-10-30T06:24:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_pq_min_cost": "65536", 2023-10-30T06:24:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_public_type": "", 2023-10-30T06:24:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_service_mode": "crc secure", 2023-10-30T06:24:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_listen_backlog": "512", 2023-10-30T06:24:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_nodelay": "true", 2023-10-30T06:24:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-10-30T06:24:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_tcp_rcvbuf": "0", 2023-10-30T06:24:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ms_type": "async", 2023-10-30T06:24:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "no_config_file": "false", 2023-10-30T06:24:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_completion_locks_per_session": "32", 2023-10-30T06:24:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_debug_inject_relock_delay": "false", 2023-10-30T06:24:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_op_bytes": "104857600", 2023-10-30T06:24:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inflight_ops": "1024", 2023-10-30T06:24:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_inject_no_watch_ping": "false", 2023-10-30T06:24:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-10-30T06:24:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_tick_interval": "5.000000", 2023-10-30T06:24:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter_timeout": "10.000000", 2023-10-30T06:24:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "objectstore_blackhole": "false", 2023-10-30T06:24:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "openssl_engine_opts": "", 2023-10-30T06:24:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_delay_time": "5.000000", 2023-10-30T06:24:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_hist_halflife": "1000", 2023-10-30T06:24:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_low_ops": "2", 2023-10-30T06:24:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_max_ops": "4", 2023-10-30T06:24:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-10-30T06:24:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_quantize_effort": "0.100000", 2023-10-30T06:24:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_agent_slop": "0.020000", 2023-10-30T06:24:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-10-30T06:24:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_allow_recovery_below_min_size": "true", 2023-10-30T06:24:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_asio_thread_count": "2", 2023-10-30T06:24:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_async_recovery_min_cost": "100", 2023-10-30T06:24:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_auto_mark_unfound_lost": "false", 2023-10-30T06:24:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_retry_interval": "30.000000", 2023-10-30T06:24:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_max": "512", 2023-10-30T06:24:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backfill_scan_min": "64", 2023-10-30T06:24:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_degraded": "false", 2023-10-30T06:24:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_peering": "false", 2023-10-30T06:24:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_backoff_on_unfound": "true", 2023-10-30T06:24:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_beacon_report_interval": "300", 2023-10-30T06:24:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_duration": "30", 2023-10-30T06:24:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-10-30T06:24:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_max_block_size": "67108864", 2023-10-30T06:24:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bench_small_size_max_iops": "100", 2023-10-30T06:24:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blkin_trace_all": "false", 2023-10-30T06:24:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_blocked_scrub_grace_period": "120", 2023-10-30T06:24:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-10-30T06:24:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2023-10-30T06:24:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-10-30T06:24:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_for_log_corruption": "false", 2023-10-30T06:24:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-10-30T06:24:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-10-30T06:24:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-10-30T06:24:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-10-30T06:24:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_class_update_on_start": "true", 2023-10-30T06:24:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_cap": "256", 2023-10-30T06:24:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_message_size_cap": "524288000", 2023-10-30T06:24:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_op_priority": "63", 2023-10-30T06:24:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_client_watch_timeout": "30", 2023-10-30T06:24:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_command_max_records": "256", 2023-10-30T06:24:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_compact_on_start": "false", 2023-10-30T06:24:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-10-30T06:24:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_chooseleaf_type": "0", 2023-10-30T06:24:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_initial_weight": "-1.000000", 2023-10-30T06:24:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_on_start": "true", 2023-10-30T06:24:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_crush_update_weight_set": "true", 2023-10-30T06:24:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-10-30T06:24:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-10-30T06:24:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-10-30T06:24:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_duration": "0", 2023-10-30T06:24:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-10-30T06:24:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_feed_pullee": "-1", 2023-10-30T06:24:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-10-30T06:24:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-10-30T06:24:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_misdirected_ops": "false", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_acting_change": "false", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_no_purge_strays": "false", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_op_order": "false", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pg_log_writeout": "false", 2023-10-30T06:24:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_pretend_recovery_active": "false", 2023-10-30T06:24:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-10-30T06:24:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-10-30T06:24:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_shutdown": "false", 2023-10-30T06:24:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-10-30T06:24:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-10-30T06:24:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_cached_snaps": "false", 2023-10-30T06:24:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_missing_on_start": "false", 2023-10-30T06:24:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_snaps": "false", 2023-10-30T06:24:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-10-30T06:24:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-10-30T06:24:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_keys": "1024", 2023-10-30T06:24:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-10-30T06:24:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-10-30T06:24:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-10-30T06:24:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_stride": "524288", 2023-10-30T06:24:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-10-30T06:24:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_data_pool_replay_window": "5", 2023-10-30T06:24:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_default_notify_timeout": "30", 2023-10-30T06:24:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep": "0.000000", 2023-10-30T06:24:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-10-30T06:24:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-10-30T06:24:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-10-30T06:24:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_discard_disconnected_ops": "true", 2023-10-30T06:24:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_enable_op_tracker": "true", 2023-10-30T06:24:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-10-30T06:24:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_extblkdev_plugins": "vdo", 2023-10-30T06:24:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-10-30T06:24:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-10-30T06:24:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_info": "true", 2023-10-30T06:24:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown": "true", 2023-10-30T06:24:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-10-30T06:24:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_fast_shutdown_timeout": "15", 2023-10-30T06:24:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-10-30T06:24:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-10-30T06:24:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-10-30T06:24:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_function_tracing": "false", 2023-10-30T06:24:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_grace": "80", 2023-10-30T06:24:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_interval": "6", 2023-10-30T06:24:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-10-30T06:24:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_peers": "10", 2023-10-30T06:24:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_min_size": "2000", 2023-10-30T06:24:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_stale": "600", 2023-10-30T06:24:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-10-30T06:24:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_max_size": "100000", 2023-10-30T06:24:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_min_size": "1000", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-10-30T06:24:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_flush_on_shutdown": "true", 2023-10-30T06:24:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_journal_size": "5120", 2023-10-30T06:24:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_kill_backfill_at": "0", 2023-10-30T06:24:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_loop_before_reset_tphandle": "64", 2023-10-30T06:24:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_cache_size": "50", 2023-10-30T06:24:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_dedup": "true", 2023-10-30T06:24:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max": "40", 2023-10-30T06:24:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_message_max_bytes": "10485760", 2023-10-30T06:24:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_map_share_max_epochs": "40", 2023-10-30T06:24:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_name_len": "100", 2023-10-30T06:24:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_attr_size": "0", 2023-10-30T06:24:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_backfills": "1", 2023-10-30T06:24:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_count": "5", 2023-10-30T06:24:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_markdown_period": "600", 2023-10-30T06:24:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_name_len": "2048", 2023-10-30T06:24:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_namespace_len": "256", 2023-10-30T06:24:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_object_size": "134217728", 2023-10-30T06:24:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-10-30T06:24:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_omap_entries_per_request": "1024", 2023-10-30T06:24:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_blocked_by": "16", 2023-10-30T06:24:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_log_entries": "10000", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_pgls": "1024", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_cost": "8388608", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_push_objects": "10", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_scrubs": "3", 2023-10-30T06:24:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-10-30T06:24:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_trimming_pgs": "2", 2023-10-30T06:24:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_op_reply_len": "64", 2023-10-30T06:24:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_max_write_size": "90", 2023-10-30T06:24:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2023-10-30T06:24:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2023-10-30T06:24:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2023-10-30T06:24:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2023-10-30T06:24:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2023-10-30T06:24:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2023-10-30T06:24:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2023-10-30T06:24:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_override_recovery_settings": "false", 2023-10-30T06:24:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_profile": "balanced", 2023-10-30T06:24:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2023-10-30T06:24:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2023-10-30T06:24:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2023-10-30T06:24:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2023-10-30T06:24:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_scheduler_client_wgt": "1", 2023-10-30T06:24:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mclock_skip_benchmark": "false", 2023-10-30T06:24:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_base": "805306368", 2023-10-30T06:24:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_min": "134217728", 2023-10-30T06:24:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_cache_resize_interval": "1.000000", 2023-10-30T06:24:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_expected_fragmentation": "0.150000", 2023-10-30T06:24:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target": "4294967296", 2023-10-30T06:24:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_autotune": "false", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_pg_log_entries": "250", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_min_recovery_priority": "0", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_interval": "30", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2023-10-30T06:24:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_interval": "5", 2023-10-30T06:24:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_report_max_in_flight": "2", 2023-10-30T06:24:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mon_shutdown_timeout": "5.000000", 2023-10-30T06:24:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_cache_shards": "32", 2023-10-30T06:24:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_num_op_tracker_shard": "32", 2023-10-30T06:24:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_auto_affinity": "true", 2023-10-30T06:24:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_node": "-1", 2023-10-30T06:24:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_numa_prefer_iface": "true", 2023-10-30T06:24:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_object_clean_region_max_num_intervals": "10", 2023-10-30T06:24:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objecter_finishers": "1", 2023-10-30T06:24:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore": "bluestore", 2023-10-30T06:24:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_fuse": "false", 2023-10-30T06:24:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_objectstore_tracing": "false", 2023-10-30T06:24:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_complaint_time": "30.000000", 2023-10-30T06:24:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_duration": "600", 2023-10-30T06:24:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_size": "20", 2023-10-30T06:24:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_size": "20", 2023-10-30T06:24:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2023-10-30T06:24:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_log_threshold": "5", 2023-10-30T06:24:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards": "0", 2023-10-30T06:24:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_hdd": "5", 2023-10-30T06:24:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_shards_ssd": "8", 2023-10-30T06:24:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard": "0", 2023-10-30T06:24:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2023-10-30T06:24:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2023-10-30T06:24:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2023-10-30T06:24:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_pq_min_cost": "65536", 2023-10-30T06:24:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue": "mclock_scheduler", 2023-10-30T06:24:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_queue_cut_off": "high", 2023-10-30T06:24:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_suicide_timeout": "150", 2023-10-30T06:24:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_op_thread_timeout": "15", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_open_classes_on_start": "true", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_os_flags": "0", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_peering_op_priority": "255", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_cost": "1048576", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_delete_priority": "5", 2023-10-30T06:24:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2023-10-30T06:24:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2023-10-30T06:24:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_dups_tracked": "3000", 2023-10-30T06:24:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_max": "10000", 2023-10-30T06:24:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_log_trim_min": "100", 2023-10-30T06:24:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2023-10-30T06:24:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_object_context_cache_count": "64", 2023-10-30T06:24:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pg_stat_report_interval_max": "500", 2023-10-30T06:24:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2023-10-30T06:24:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_evict_age": "0", 2023-10-30T06:24:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_min_flush_age": "0", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crimson": "false", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_crush_rule": "-1", 2023-10-30T06:24:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_ec_fast_read": "false", 2023-10-30T06:24:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2023-10-30T06:24:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_bulk": "false", 2023-10-30T06:24:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_hashpspool": "true", 2023-10-30T06:24:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nodelete": "false", 2023-10-30T06:24:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nopgchange": "false", 2023-10-30T06:24:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flag_nosizechange": "false", 2023-10-30T06:24:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_flags": "0", 2023-10-30T06:24:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2023-10-30T06:24:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_min_size": "0", 2023-10-30T06:24:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2023-10-30T06:24:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pg_num": "32", 2023-10-30T06:24:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_pgp_num": "0", 2023-10-30T06:24:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2023-10-30T06:24:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_size": "2", 2023-10-30T06:24:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_default_type": "replicated", 2023-10-30T06:24:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2023-10-30T06:24:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pool_use_gmt_hitset": "true", 2023-10-30T06:24:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_push_per_object_cost": "1000", 2023-10-30T06:24:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_read_ec_check_for_errors": "false", 2023-10-30T06:24:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap": "true", 2023-10-30T06:24:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recover_clone_overlap_limit": "10", 2023-10-30T06:24:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_cost": "20971520", 2023-10-30T06:24:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_delay_start": "0.000000", 2023-10-30T06:24:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active": "0", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_hdd": "3", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_active_ssd": "10", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_chunk": "8388608", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_max_single_start": "1", 2023-10-30T06:24:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_priority": "3", 2023-10-30T06:24:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_op_warn_multiple": "16", 2023-10-30T06:24:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_priority": "5", 2023-10-30T06:24:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_retry_interval": "30.000000", 2023-10-30T06:24:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep": "0.000000", 2023-10-30T06:24:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hdd": "0.100000", 2023-10-30T06:24:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2023-10-30T06:24:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_recovery_sleep_ssd": "0.000000", 2023-10-30T06:24:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_repair_during_recovery": "false", 2023-10-30T06:24:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_requested_scrub_priority": "120", 2023-10-30T06:24:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2023-10-30T06:24:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_rollback_to_cluster_snap": "", 2023-10-30T06:24:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair": "false", 2023-10-30T06:24:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_auto_repair_num_errors": "5", 2023-10-30T06:24:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_backoff_ratio": "0.660000", 2023-10-30T06:24:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_hour": "0", 2023-10-30T06:24:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_begin_week_day": "0", 2023-10-30T06:24:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_max": "25", 2023-10-30T06:24:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_chunk_min": "5", 2023-10-30T06:24:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_cost": "52428800", 2023-10-30T06:24:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_during_recovery": "false", 2023-10-30T06:24:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_hour": "0", 2023-10-30T06:24:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_end_week_day": "0", 2023-10-30T06:24:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_event_cost": "4096", 2023-10-30T06:24:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_extended_sleep": "0.000000", 2023-10-30T06:24:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_invalid_stats": "true", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_load_threshold": "0.500000", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_interval": "604800.000000", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_max_preemptions": "5", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_min_interval": "86400.000000", 2023-10-30T06:24:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_priority": "5", 2023-10-30T06:24:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_reservation_timeout": "5000", 2023-10-30T06:24:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_sleep": "0.000000", 2023-10-30T06:24:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_scrub_slow_reservation_response": "2200", 2023-10-30T06:24:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_max": "100", 2023-10-30T06:24:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shallow_scrub_chunk_min": "50", 2023-10-30T06:24:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_shutdown_pgref_assert": "false", 2023-10-30T06:24:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_skip_data_digest": "false", 2023-10-30T06:24:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_smart_report_timeout": "5", 2023-10-30T06:24:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_cost": "1048576", 2023-10-30T06:24:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_priority": "5", 2023-10-30T06:24:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep": "0.000000", 2023-10-30T06:24:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2023-10-30T06:24:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2023-10-30T06:24:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2023-10-30T06:24:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_not_scrubbing": "120", 2023-10-30T06:24:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_stats_update_period_scrubbing": "15", 2023-10-30T06:24:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2023-10-30T06:24:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_target_transaction_size": "30", 2023-10-30T06:24:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_count": "4", 2023-10-30T06:24:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2023-10-30T06:24:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2023-10-30T06:24:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2023-10-30T06:24:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2023-10-30T06:24:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2023-10-30T06:24:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2023-10-30T06:24:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_default_cache_mode": "writeback", 2023-10-30T06:24:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2023-10-30T06:24:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tier_promote_max_objects_sec": "25", 2023-10-30T06:24:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_tracing": "false", 2023-10-30T06:24:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_use_stale_snap": "false", 2023-10-30T06:24:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2023-10-30T06:24:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "osdc_blkin_trace_all": "false", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_kill_at": "0", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_max_join_drift": "10", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min": "500", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_min_wait": "0.050000", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_propose_interval": "1.000000", 2023-10-30T06:24:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max": "500", 2023-10-30T06:24:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_max_multiplier": "20", 2023-10-30T06:24:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_service_trim_min": "10", 2023-10-30T06:24:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_stash_full_interval": "25", 2023-10-30T06:24:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_max": "500", 2023-10-30T06:24:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "paxos_trim_min": "250", 2023-10-30T06:24:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "perf": "true", 2023-10-30T06:24:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2023-10-30T06:24:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_crypto_accelerator": "crypto_isal", 2023-10-30T06:24:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin_dir": "/usr/lib64/ceph", 2023-10-30T06:24:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "-", 2023-10-30T06:24:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrv": "", 2023-10-30T06:24:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_bind_addr": "-", 2023-10-30T06:24:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network": "", 2023-10-30T06:24:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "public_network_interface": "", 2023-10-30T06:24:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_enabled": "false", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "qat_compressor_session_max_number": "256", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_mon_op_timeout": "0", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_osd_op_timeout": "0", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rados_tracing": "false", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_atime_update_interval": "60", 2023-10-30T06:24:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2023-10-30T06:24:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_parent_reads": "false", 2023-10-30T06:24:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_balance_snap_reads": "false", 2023-10-30T06:24:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blkin_trace_all": "false", 2023-10-30T06:24:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_expire_seconds": "0", 2023-10-30T06:24:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_blocklist_on_break_lock": "true", 2023-10-30T06:24:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache": "true", 2023-10-30T06:24:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_block_writes_upfront": "false", 2023-10-30T06:24:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty": "25165824", 2023-10-30T06:24:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_age": "1.000000", 2023-10-30T06:24:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_max_dirty_object": "0", 2023-10-30T06:24:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_policy": "writearound", 2023-10-30T06:24:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_size": "33554432", 2023-10-30T06:24:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_target_dirty": "16777216", 2023-10-30T06:24:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_cache_writethrough_until_flush": "true", 2023-10-30T06:24:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_clone_copy_on_read": "false", 2023-10-30T06:24:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_compression_hint": "none", 2023-10-30T06:24:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_concurrent_management_ops": "10", 2023-10-30T06:24:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_config_pool_override_update_timestamp": "0", 2023-10-30T06:24:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_clone_format": "auto", 2023-10-30T06:24:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_data_pool": "", 2023-10-30T06:24:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_features": "61", 2023-10-30T06:24:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_format": "2", 2023-10-30T06:24:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_map_options": "", 2023-10-30T06:24:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_order": "22", 2023-10-30T06:24:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_pool": "rbd", 2023-10-30T06:24:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2023-10-30T06:24:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_count": "0", 2023-10-30T06:24:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_default_stripe_unit": "0", 2023-10-30T06:24:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_disable_zero_copy_writes": "true", 2023-10-30T06:24:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_granularity_bytes": "65536", 2023-10-30T06:24:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_discard_on_zeroed_write_same": "true", 2023-10-30T06:24:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_enable_alloc_hint": "true", 2023-10-30T06:24:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2023-10-30T06:24:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler": "simple", 2023-10-30T06:24:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2023-10-30T06:24:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_commit_age": "5.000000", 2023-10-30T06:24:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2023-10-30T06:24:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_max_payload_bytes": "16384", 2023-10-30T06:24:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_age": "0.000000", 2023-10-30T06:24:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_bytes": "1048576", 2023-10-30T06:24:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_flush_interval": "0", 2023-10-30T06:24:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2023-10-30T06:24:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2023-10-30T06:24:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_order": "24", 2023-10-30T06:24:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_pool": "", 2023-10-30T06:24:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_journal_splay_width": "4", 2023-10-30T06:24:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_parent_reads": "false", 2023-10-30T06:24:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_localize_snap_reads": "false", 2023-10-30T06:24:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2023-10-30T06:24:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2023-10-30T06:24:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2023-10-30T06:24:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2023-10-30T06:24:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2023-10-30T06:24:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2023-10-30T06:24:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_type": "simple", 2023-10-30T06:24:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2023-10-30T06:24:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_image_state_check_interval": "30", 2023-10-30T06:24:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2023-10-30T06:24:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2023-10-30T06:24:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2023-10-30T06:24:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2023-10-30T06:24:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2023-10-30T06:24:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_autotune": "true", 2023-10-30T06:24:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_base": "805306368", 2023-10-30T06:24:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2023-10-30T06:24:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_min": "134217728", 2023-10-30T06:24:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2023-10-30T06:24:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2023-10-30T06:24:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_memory_target": "4294967296", 2023-10-30T06:24:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_perf_stats_prio": "5", 2023-10-30T06:24:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2023-10-30T06:24:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_delete_delay": "0", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_replay_delay": "0", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2023-10-30T06:24:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove": "false", 2023-10-30T06:24:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2023-10-30T06:24:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_mtime_update_interval": "60", 2023-10-30T06:24:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_non_blocking_aio": "true", 2023-10-30T06:24:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_thread_timeout": "60", 2023-10-30T06:24:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_op_threads": "1", 2023-10-30T06:24:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_parent_cache_enabled": "false", 2023-10-30T06:24:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_mode": "disabled", 2023-10-30T06:24:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_path": "/tmp", 2023-10-30T06:24:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_persistent_cache_size": "1073741824", 2023-10-30T06:24:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_plugins": "", 2023-10-30T06:24:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst": "0", 2023-10-30T06:24:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_burst_seconds": "1", 2023-10-30T06:24:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_bps_limit": "0", 2023-10-30T06:24:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_exclude_ops": "0", 2023-10-30T06:24:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst": "0", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_burst_seconds": "1", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_iops_limit": "0", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst": "0", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_bps_limit": "0", 2023-10-30T06:24:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst": "0", 2023-10-30T06:24:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2023-10-30T06:24:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_read_iops_limit": "0", 2023-10-30T06:24:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_schedule_tick_min": "50", 2023-10-30T06:24:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst": "0", 2023-10-30T06:24:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2023-10-30T06:24:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_bps_limit": "0", 2023-10-30T06:24:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst": "0", 2023-10-30T06:24:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2023-10-30T06:24:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_qos_write_iops_limit": "0", 2023-10-30T06:24:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_quiesce_notification_attempts": "10", 2023-10-30T06:24:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_read_from_replica_policy": "default", 2023-10-30T06:24:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2023-10-30T06:24:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_max_bytes": "524288", 2023-10-30T06:24:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_readahead_trigger_requests": "10", 2023-10-30T06:24:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_request_timed_out_seconds": "30", 2023-10-30T06:24:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_skip_partial_discard": "true", 2023-10-30T06:24:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2023-10-30T06:24:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_tracing": "false", 2023-10-30T06:24:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_names": "true", 2023-10-30T06:24:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd_validate_pool": "true", 2023-10-30T06:24:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_base_url": "", 2023-10-30T06:24:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "restapi_log_level": "", 2023-10-30T06:24:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_acl_grants_max_num": "100", 2023-10-30T06:24:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_admin_entry": "admin", 2023-10-30T06:24:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2023-10-30T06:24:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_backend_store": "rados", 2023-10-30T06:24:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_barbican_url": "", 2023-10-30T06:24:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_beast_enable_async": "true", 2023-10-30T06:24:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_counters_cache": "false", 2023-10-30T06:24:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_counters_cache_size": "10000", 2023-10-30T06:24:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2023-10-30T06:24:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_default_quota_max_size": "-1", 2023-10-30T06:24:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_max_aio": "128", 2023-10-30T06:24:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2023-10-30T06:24:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_cache_size": "10000", 2023-10-30T06:24:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_quota_ttl": "600", 2023-10-30T06:24:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_bucket_sync_spawn_window": "20", 2023-10-30T06:24:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_enabled": "true", 2023-10-30T06:24:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_expiry_interval": "900", 2023-10-30T06:24:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cache_lru_size": "10000", 2023-10-30T06:24:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_config_store": "rados", 2023-10-30T06:24:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_content_length_compat": "false", 2023-10-30T06:24:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress": "true", 2023-10-30T06:24:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2023-10-30T06:24:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cors_rules_max_num": "100", 2023-10-30T06:24:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_cross_domain_policy": "", 2023-10-30T06:24:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_default_encryption_key": "", 2023-10-30T06:24:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_addr": "", 2023-10-30T06:24:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_ca_path": "", 2023-10-30T06:24:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_cert": "", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_client_key": "", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_kms_key_template": "", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_password": "", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_kmip_username": "", 2023-10-30T06:24:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_require_ssl": "true", 2023-10-30T06:24:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2023-10-30T06:24:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2023-10-30T06:24:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_backend": "vault", 2023-10-30T06:24:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2023-10-30T06:24:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2023-10-30T06:24:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2023-10-30T06:24:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2023-10-30T06:24:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2023-10-30T06:24:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2023-10-30T06:24:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2023-10-30T06:24:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2023-10-30T06:24:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2023-10-30T06:24:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2023-10-30T06:24:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2023-10-30T06:24:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_suppress_logs": "true", 2023-10-30T06:24:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_addr": "", 2023-10-30T06:24:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_auth": "token", 2023-10-30T06:24:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_namespace": "", 2023-10-30T06:24:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_prefix": "", 2023-10-30T06:24:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_secret_engine": "transit", 2023-10-30T06:24:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_cacert": "", 2023-10-30T06:24:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2023-10-30T06:24:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2023-10-30T06:24:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_token_file": "", 2023-10-30T06:24:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_crypt_vault_verify_ssl": "true", 2023-10-30T06:24:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_buffersize": "524288", 2023-10-30T06:24:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_limit": "1024", 2023-10-30T06:24:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_low_speed_time": "300", 2023-10-30T06:24:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_tcp_keepalive": "0", 2023-10-30T06:24:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_curl_wait_timeout_ms": "1000", 2023-10-30T06:24:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2023-10-30T06:24:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2023-10-30T06:24:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2023-10-30T06:24:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2023-10-30T06:24:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_fadvise": "4", 2023-10-30T06:24:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2023-10-30T06:24:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_num": "64", 2023-10-30T06:24:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d3n_libaio_aio_threads": "20", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d4n_host": "127.0.0.1", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_d4n_port": "6379", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_changes_size": "1000", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_num_shards": "128", 2023-10-30T06:24:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_obj_prefix": "data_log", 2023-10-30T06:24:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_log_window": "30", 2023-10-30T06:24:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_notify_interval_msec": "0", 2023-10-30T06:24:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_poll_interval": "20", 2023-10-30T06:24:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_data_sync_spawn_window": "20", 2023-10-30T06:24:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_debug_inject_set_olh_err": "0", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_data_log_backing": "fifo", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_realm_info_oid": "default.realm", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_region_info_oid": "default.region", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zone_info_oid": "default.zone", 2023-10-30T06:24:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2023-10-30T06:24:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_defer_to_bucket_acls": "", 2023-10-30T06:24:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_delete_multi_obj_max_num": "1000", 2023-10-30T06:24:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_lim": "0.000000", 2023-10-30T06:24:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_res": "100.000000", 2023-10-30T06:24:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2023-10-30T06:24:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_lim": "0.000000", 2023-10-30T06:24:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_res": "200.000000", 2023-10-30T06:24:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2023-10-30T06:24:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_lim": "0.000000", 2023-10-30T06:24:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_res": "500.000000", 2023-10-30T06:24:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_data_wgt": "500.000000", 2023-10-30T06:24:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2023-10-30T06:24:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_res": "500.000000", 2023-10-30T06:24:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2023-10-30T06:24:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_name": "", 2023-10-30T06:24:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dns_s3website_name": "", 2023-10-30T06:24:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_dynamic_resharding": "true", 2023-10-30T06:24:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2023-10-30T06:24:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_gc_threads": "true", 2023-10-30T06:24:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_lc_threads": "true", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_ops_log": "false", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_quota_threads": "true", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_static_website": "false", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enable_usage_log": "false", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_enforce_swift_acls": "true", 2023-10-30T06:24:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_exit_timeout_secs": "120", 2023-10-30T06:24:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_expose_bucket": "false", 2023-10-30T06:24:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_extended_http_attrs": "", 2023-10-30T06:24:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_filter": "none", 2023-10-30T06:24:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2023-10-30T06:24:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_frontends": "beast port=7480", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_concurrent_io": "10", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred": "50", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_objs": "32", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_queue_size": "134213632", 2023-10-30T06:24:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_max_trim_chunk": "16", 2023-10-30T06:24:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_obj_min_wait": "7200", 2023-10-30T06:24:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_max_time": "3600", 2023-10-30T06:24:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_gc_processor_period": "3600", 2023-10-30T06:24:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_max_req_size": "4194304", 2023-10-30T06:24:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_get_obj_window_size": "16777216", 2023-10-30T06:24:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_healthcheck_disabling_path": "", 2023-10-30T06:24:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ignore_get_invalid_range": "false", 2023-10-30T06:24:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_init_timeout": "300", 2023-10-30T06:24:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2023-10-30T06:24:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_admin_roles": "", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_reader_roles": "", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_domain": "", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password": "", 2023-10-30T06:24:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_password_path": "", 2023-10-30T06:24:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_project": "", 2023-10-30T06:24:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_tenant": "", 2023-10-30T06:24:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token": "", 2023-10-30T06:24:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_token_path": "", 2023-10-30T06:24:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_admin_user": "", 2023-10-30T06:24:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_api_version": "2", 2023-10-30T06:24:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_domain": "", 2023-10-30T06:24:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_password": "", 2023-10-30T06:24:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_project": "", 2023-10-30T06:24:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_tenant": "", 2023-10-30T06:24:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_barbican_user": "", 2023-10-30T06:24:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2023-10-30T06:24:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_implicit_tenants": "false", 2023-10-30T06:24:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2023-10-30T06:24:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_service_token_enabled": "false", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_token_cache_size": "10000", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_url": "", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_keystone_verify_ssl": "true", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_debug_interval": "-1", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_lock_max_time": "90", 2023-10-30T06:24:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_objs": "32", 2023-10-30T06:24:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_rules": "1000", 2023-10-30T06:24:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_worker": "3", 2023-10-30T06:24:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_max_wp_worker": "3", 2023-10-30T06:24:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lc_thread_delay": "0", 2023-10-30T06:24:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2023-10-30T06:24:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_dnattr": "uid", 2023-10-30T06:24:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2023-10-30T06:24:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_searchfilter": "", 2023-10-30T06:24:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2023-10-30T06:24:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ldap_uri": "ldaps://", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_bucket_min_readahead": "1000", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_list_buckets_max_chunk": "1000", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_http_headers": "", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_nonexistent_bucket": "false", 2023-10-30T06:24:32.107 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2023-10-30T06:24:32.108 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_log_object_name_utc": "false", 2023-10-30T06:24:32.108 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_lua_max_memory_per_state": "128000", 2023-10-30T06:24:32.108 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2023-10-30T06:24:32.108 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_name_len": "0", 2023-10-30T06:24:32.108 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attr_size": "0", 2023-10-30T06:24:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_attrs_num_in_req": "0", 2023-10-30T06:24:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_chunk_size": "4194304", 2023-10-30T06:24:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_concurrent_requests": "1024", 2023-10-30T06:24:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2023-10-30T06:24:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_dynamic_shards": "1999", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_listing_results": "1000", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_notify_retries": "10", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_objs_per_shard": "100000", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_param_size": "1048576", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_put_size": "5368709120", 2023-10-30T06:24:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_max_slo_entries": "1000", 2023-10-30T06:24:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_log_max_shards": "64", 2023-10-30T06:24:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_md_notify_interval_msec": "200", 2023-10-30T06:24:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_poll_interval": "20", 2023-10-30T06:24:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_meta_sync_spawn_window": "20", 2023-10-30T06:24:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mime_types_file": "/etc/mime.types", 2023-10-30T06:24:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_mp_lock_max_time": "600", 2023-10-30T06:24:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multi_obj_del_max_aio": "16", 2023-10-30T06:24:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_min_part_size": "5242880", 2023-10-30T06:24:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_multipart_part_upload_limit": "10000", 2023-10-30T06:24:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_partitions": "3", 2023-10-30T06:24:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_fhcache_size": "2017", 2023-10-30T06:24:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_frontends": "rgw-nfs", 2023-10-30T06:24:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2023-10-30T06:24:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_lru_lanes": "5", 2023-10-30T06:24:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_max_gc": "300", 2023-10-30T06:24:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_namespace_expire_secs": "300", 2023-10-30T06:24:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_gc_threads": "false", 2023-10-30T06:24:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_lc_threads": "false", 2023-10-30T06:24:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_quota_threads": "false", 2023-10-30T06:24:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_run_sync_thread": "false", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_s3_fast_attrs": "false", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_nfs_write_completion_interval_s": "10", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_async_rados_threads": "32", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_num_control_oids": "8", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_numa_node": "-1", 2023-10-30T06:24:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_stripe_size": "4194304", 2023-10-30T06:24:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_obj_tombstone_cache_size": "1000", 2023-10-30T06:24:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_chunk_size": "100", 2023-10-30T06:24:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_gc_interval": "600", 2023-10-30T06:24:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_objexp_hints_num_shards": "127", 2023-10-30T06:24:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_olh_pending_timeout_sec": "3600", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_suicide_timeout": "0", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_thread_timeout": "600", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_op_tracing": "false", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_token": "", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_url": "", 2023-10-30T06:24:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_opa_verify_ssl": "true", 2023-10-30T06:24:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_data_backlog": "5242880", 2023-10-30T06:24:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2023-10-30T06:24:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_rados": "false", 2023-10-30T06:24:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_ops_log_socket_path": "", 2023-10-30T06:24:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_override_bucket_index_max_shards": "0", 2023-10-30T06:24:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2023-10-30T06:24:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2023-10-30T06:24:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval": "2.000000", 2023-10-30T06:24:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_push_interval_max": "30.000000", 2023-10-30T06:24:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_period_root_pool": ".rgw.root", 2023-10-30T06:24:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_policy_reject_invalid_principals": "true", 2023-10-30T06:24:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2023-10-30T06:24:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_lanes": "3", 2023-10-30T06:24:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_lmdb_count": "3", 2023-10-30T06:24:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_max_buckets": "100", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_cache_partitions": "3", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_continue": "true", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_print_prohibited_content_length": "false", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_max_window_size": "67108864", 2023-10-30T06:24:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_put_obj_min_window_size": "16777216", 2023-10-30T06:24:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2023-10-30T06:24:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_pool_recovery_priority": "5", 2023-10-30T06:24:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rados_tracing": "false", 2023-10-30T06:24:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm": "", 2023-10-30T06:24:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_id": "", 2023-10-30T06:24:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_realm_root_pool": ".rgw.root", 2023-10-30T06:24:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region": "", 2023-10-30T06:24:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_region_root_pool": ".rgw.root", 2023-10-30T06:24:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_region_enforcement": "false", 2023-10-30T06:24:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_relaxed_s3_bucket_names": "false", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_request_uri": "", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_batch_size": "64", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_bucket_lock_duration": "360", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_max_aio": "128", 2023-10-30T06:24:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_num_logs": "16", 2023-10-30T06:24:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_reshard_thread_interval": "600", 2023-10-30T06:24:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_resolve_cname": "false", 2023-10-30T06:24:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_rest_getusage_op_compat": "false", 2023-10-30T06:24:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_run_sync_thread": "true", 2023-10-30T06:24:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_order": "sts, external, local", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_keystone": "false", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_ldap": "false", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_rados": "true", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_auth_use_sts": "false", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_client_max_sig_ver": "-1", 2023-10-30T06:24:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_s3_success_create_obj_status": "0", 2023-10-30T06:24:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_safe_max_objects_per_shard": "102400", 2023-10-30T06:24:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_scheduler_type": "throttler", 2023-10-30T06:24:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_script_uri": "", 2023-10-30T06:24:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_service_provider_name": "", 2023-10-30T06:24:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_shard_warning_threshold": "90.000000", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_id": "", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_client_secret": "", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_entry": "sts", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_key": "sts", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_max_session_duration": "43200", 2023-10-30T06:24:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_min_session_duration": "900", 2023-10-30T06:24:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sts_token_introspection_url": "", 2023-10-30T06:24:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_account_in_url": "false", 2023-10-30T06:24:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_entry": "auth", 2023-10-30T06:24:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_auth_url": "", 2023-10-30T06:24:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_custom_header": "", 2023-10-30T06:24:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_enforce_content_length": "false", 2023-10-30T06:24:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_need_stats": "true", 2023-10-30T06:24:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_tenant_name": "", 2023-10-30T06:24:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_token_expiration": "86400", 2023-10-30T06:24:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url": "", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_url_prefix": "swift", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_swift_versioning_enabled": "false", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_lease_period": "120", 2023-10-30T06:24:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2023-10-30T06:24:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_interval": "1200", 2023-10-30T06:24:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_max_buckets": "16", 2023-10-30T06:24:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2023-10-30T06:24:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2023-10-30T06:24:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_obj_etag_verify": "false", 2023-10-30T06:24:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_history_size": "4096", 2023-10-30T06:24:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_per_node_log_size": "32", 2023-10-30T06:24:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2023-10-30T06:24:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_thread_pool_size": "512", 2023-10-30T06:24:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_max_retries": "0", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_sleep_duration": "0", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_topic_persistency_time_to_live": "0", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_comment": "", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_createby": "", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_encoding": "", 2023-10-30T06:24:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_flag": "false", 2023-10-30T06:24:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_max_size": "5368709120", 2023-10-30T06:24:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_origin": "", 2023-10-30T06:24:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_sha_unit": "524288", 2023-10-30T06:24:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_torrent_tracker": "", 2023-10-30T06:24:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_trust_forwarded_https": "false", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_flush_threshold": "1024", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_log_tick_interval": "30", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_shards": "32", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_usage_max_user_shards": "1", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_use_opa_authz": "false", 2023-10-30T06:24:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_counters_cache": "false", 2023-10-30T06:24:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_counters_cache_size": "10000", 2023-10-30T06:24:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_objects": "-1", 2023-10-30T06:24:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_default_quota_max_size": "-1", 2023-10-30T06:24:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_max_buckets": "1000", 2023-10-30T06:24:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_policies_max_num": "100", 2023-10-30T06:24:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2023-10-30T06:24:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_idle_users": "false", 2023-10-30T06:24:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_interval": "86400", 2023-10-30T06:24:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_quota_sync_wait_time": "86400", 2023-10-30T06:24:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_user_unique_email": "true", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_verify_ssl": "true", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_website_routing_rules_max_num": "50", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone": "", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_id": "", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zone_root_pool": ".rgw.root", 2023-10-30T06:24:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup": "", 2023-10-30T06:24:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_id": "", 2023-10-30T06:24:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2023-10-30T06:24:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_auto_session_clear": "true", 2023-10-30T06:24:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "rgwlc_skip_bucket_step": "false", 2023-10-30T06:24:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_block_size": "4096", 2023-10-30T06:24:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_bloom_bits_per_key": "20", 2023-10-30T06:24:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2023-10-30T06:24:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2023-10-30T06:24:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_row_ratio": "0.000000", 2023-10-30T06:24:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_shard_bits": "4", 2023-10-30T06:24:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_size": "536870912", 2023-10-30T06:24:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cache_type": "binned_lru", 2023-10-30T06:24:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion": "true", 2023-10-30T06:24:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2023-10-30T06:24:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_compaction_stats": "false", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_extended_stats": "false", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_collect_memory_stats": "false", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_delete_range_threshold": "1048576", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_index_type": "binary_search", 2023-10-30T06:24:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_log_to_ceph_log": "true", 2023-10-30T06:24:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_metadata_block_size": "4096", 2023-10-30T06:24:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_partition_filters": "false", 2023-10-30T06:24:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_perf": "false", 2023-10-30T06:24:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2023-10-30T06:24:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_bootstrap_timeout": "30", 2023-10-30T06:24:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "rotating_keys_renewal_timeout": "10", 2023-10-30T06:24:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "run_dir": "/var/run/ceph", 2023-10-30T06:24:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_block_create": "true", 2023-10-30T06:24:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cache_lru_size": "67108864", 2023-10-30T06:24:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_cbjournal_size": "5368709120", 2023-10-30T06:24:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_max_object_size": "16777216", 2023-10-30T06:24:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_default_object_metadata_reservation": "16777216", 2023-10-30T06:24:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_device_size": "53687091200", 2023-10-30T06:24:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_capacity": "16", 2023-10-30T06:24:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_flush_size": "16777216", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_journal_iodepth_limit": "5", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_main_device_type": "SSD", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_max_concurrent_transactions": "8", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2023-10-30T06:24:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2023-10-30T06:24:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2023-10-30T06:24:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_obj_data_write_amplification": "1.250000", 2023-10-30T06:24:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "seastore_segment_size": "67108864", 2023-10-30T06:24:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "set_keepcaps": "false", 2023-10-30T06:24:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "setgroup": "", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser": "", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "setuser_match_path": "", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_misplaced_ratio": "0.050000", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "thp": "false", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_default_timeout": "60", 2023-10-30T06:24:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "threadpool_empty_queue_max_wait": "2", 2023-10-30T06:24:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "throttler_perf_counter": "true" 2023-10-30T06:24:32.150 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:32.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2023-10-30T06:24:32.392 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2023-10-30T06:24:32.393 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2023-10-30T06:24:32.488 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2023-10-30T06:24:32.488 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2023-10-30T06:24:32.582 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2023-10-30T06:24:32.583 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2023-10-30T06:24:32.593 DEBUG:teuthology.orchestra.run.smithi112:> type ceph-objectstore-tool 2023-10-30T06:24:32.607 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-10-30T06:24:32.607 DEBUG:teuthology.orchestra.run.smithi079:> type ceph-objectstore-tool 2023-10-30T06:24:32.621 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-10-30T06:24:32.622 INFO:teuthology.run_tasks:Running task mon_recovery... 2023-10-30T06:24:32.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:24:32.699 INFO:tasks.thrashosds.thrasher:starting do_thrash 2023-10-30T06:24:32.699 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:24:32.699 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:24:32.700 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:24:32.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:24:32.700 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2023-10-30T06:24:32.701 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2023-10-30T06:24:32.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:24:32.702 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2023-10-30T06:24:32.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:32.702 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2023-10-30T06:24:32.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:24:32.707 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:32.704+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:32.808 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:32.804+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:32.870 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:32.870 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:32.870 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:32.870 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:32.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:32.931 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:32.927+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:33.011 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:33.008+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:33.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:24:33.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.110 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:33.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:33.120 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:33.120 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:33.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:33.211 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:33.208+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:33.265 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:24:33.266 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.308 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:33.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:33.339 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:33.339 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:33.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:33.437 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:33.436+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:33.441 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:24:33.441 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.492 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:33.492 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":24,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:24:21.220169+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:23:28.674155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:24:33.509 INFO:tasks.mon_recovery.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-30T06:24:33.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-30T06:24:33.538 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:33.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:33.551 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:33.551 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 24, 2023-10-30T06:24:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:24:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:24:21.220169+0000", 2023-10-30T06:24:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:24:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:24:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:24:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2023-10-30T06:24:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:24:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:24:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:24:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:24:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:24:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:24:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:24:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:24:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:24:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2023-10-30T06:24:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:24:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:24:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:24:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:24:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:24:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:24:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:24:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:24:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:24:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:24:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:24:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:24:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:24:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:24:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:24:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:24:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:24:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:24:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:24:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:24:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:24:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:24:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:24:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:24:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:24:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:24:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:24:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:24:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:24:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:24:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:24:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:24:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:24:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:24:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:24:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:24:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:24:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:24:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:24:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:24:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:24:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:24:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:24:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:24:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:24:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:24:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:24:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:24:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:24:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:24:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:24:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:24:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:24:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:24:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:24:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:24:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:24:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:24:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:24:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:24:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:24:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:24:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:24:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:24:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:24:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 8, 2023-10-30T06:24:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 8, 2023-10-30T06:24:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.25, 2023-10-30T06:24:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:24:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:24:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2023-10-30T06:24:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:24:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2023-10-30T06:24:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:24:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:24:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:24:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:24:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:24:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:24:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:24:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:24:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:24:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:24:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:24:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:24:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:24:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:24:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:24:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:24:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:24:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:24:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:24:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:24:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:24:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:24:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:24:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:24:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:24:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:24:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:24:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:24:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:24:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:24:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:24:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:24:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:24:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:24:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:24:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:24:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:24:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:24:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:24:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:24:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:24:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:24:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:24:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:24:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:24:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:24:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:24:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:24:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:24:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:24:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:24:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:24:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:24:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:24:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:24:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:24:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:24:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:24:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:24:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:24:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 4.9800000190734863, 2023-10-30T06:24:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 4.9800000190734863, 2023-10-30T06:24:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.87999999523162842, 2023-10-30T06:24:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 4.380000114440918, 2023-10-30T06:24:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 4.380000114440918, 2023-10-30T06:24:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2023-10-30T06:24:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:24:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2023-10-30T06:24:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:24:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:24:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:24:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:24:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:24:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:24:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:24:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:24:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:24:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:24:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:24:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:24:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:24:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:24:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:24:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 18, 2023-10-30T06:24:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:24:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:24:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:24:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:24:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:24:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:24:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:24:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:24:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:24:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:24:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:24:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:24:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:24:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:24:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:24:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:24:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:24:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:24:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:24:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:24:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:24:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:24:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:24:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:24:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:24:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:24:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:24:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:24:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:24:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:24:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:24:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:24:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:24:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:24:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:24:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:24:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:24:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:24:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:24:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:24:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:24:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:24:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:24:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:24:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:24:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:24:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:24:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:24:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:24:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:24:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:24:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:24:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:24:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:24:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:24:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:24:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:24:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:24:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:24:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:24:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:24:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:24:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:24:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:24:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:24:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:24:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:24:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:24:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:24:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:24:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:24:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:24:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:24:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:24:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:24:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:24:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:24:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:24:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:24:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:24:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:24:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:24:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:24:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:24:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:24:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:24:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:24:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:24:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:24:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:24:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:24:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:24:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:24:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:24:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:24:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:24:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:24:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:24:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:24:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:24:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:24:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2023-10-30T06:24:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2023-10-30T06:24:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:24:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:24:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:24:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:24:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:24:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:24:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:24:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:24:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:24:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:24:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:24:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:24:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:24:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:33.756 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:33.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:33.759 INFO:tasks.thrashosds.thrasher:j is {'epoch': 24, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:24:21.220169+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:24:33.760 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2023-10-30T06:24:33.760 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:24:33.760 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.761 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:24:33.761 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.764 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:33.762+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:33.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:24:33.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:24:33.858 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:33.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:33.895 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:33.896 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:33.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:33.959 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:33.959+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:33.970 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:24:33.971 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:24:34.051 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.052 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:34.061 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.112 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:24:34.113 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:24:34.149 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-10-30T06:24:34.161 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.161+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-30T06:24:34.246 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.246 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:34.262 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:34.261+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:34.354 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:24:34.363 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:34.362+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:34.442 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.442 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:34.497 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:34.496+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:34.565 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.564+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.581 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.581 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:34.676 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.675+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.766 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:34.765+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:34.771 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.771 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:34.867 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.953 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:34.954 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:34.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:34.983 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-10-30T06:24:34.984 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:34.983+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:34.997 INFO:tasks.mon_recovery:mon ids = ['a', 'c', 'b'] 2023-10-30T06:24:34.997 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:24:34.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:24:35.069 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:35.068+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:35.113 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.113 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:35.170 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:35.170+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:35.259 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.260 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:35.271 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:35.270+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:35.372 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:35.370+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:35.415 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.416 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:35.511 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:35.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:35.574 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:35.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:35.595 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.595 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:35.674 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:35.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:35.692 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":72,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:24:35.706 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:24:35.706 INFO:tasks.mon_recovery:verifying all monitors are in the quorum 2023-10-30T06:24:35.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2023-10-30T06:24:35.775 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:35.774+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:35.780 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.780 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:35.876 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:35.875+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:35.958 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:35.958 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:35.960 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:35.960 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "a", 2023-10-30T06:24:35.960 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 0, 2023-10-30T06:24:35.961 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "leader", 2023-10-30T06:24:35.961 INFO:teuthology.orchestra.run.smithi079.stdout: "election_epoch": 8, 2023-10-30T06:24:35.961 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum": [ 2023-10-30T06:24:35.961 INFO:teuthology.orchestra.run.smithi079.stdout: 0, 2023-10-30T06:24:35.961 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:24:35.962 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:24:35.962 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:35.962 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_age": 72, 2023-10-30T06:24:35.962 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:35.962 INFO:teuthology.orchestra.run.smithi079.stdout: "required_con": "2449958758054445060", 2023-10-30T06:24:35.963 INFO:teuthology.orchestra.run.smithi079.stdout: "required_mon": [ 2023-10-30T06:24:35.963 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:35.963 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:35.963 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:35.963 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:35.964 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:35.965 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:35.965 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_con": "4540138322906710015", 2023-10-30T06:24:35.965 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_mon": [ 2023-10-30T06:24:35.965 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:35.965 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:35.966 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:35.966 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:35.966 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:35.966 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:35.966 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:35.967 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:35.967 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:35.967 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:35.967 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "outside_quorum": [], 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "extra_probe_peers": [], 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "sync_provider": [], 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": { 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 1, 2023-10-30T06:24:35.968 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:35.969 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:35.969 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:35.969 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release": 18, 2023-10-30T06:24:35.969 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release_name": "reef", 2023-10-30T06:24:35.969 INFO:teuthology.orchestra.run.smithi079.stdout: "election_strategy": 3, 2023-10-30T06:24:35.970 INFO:teuthology.orchestra.run.smithi079.stdout: "disallowed_leaders: ": "", 2023-10-30T06:24:35.970 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false, 2023-10-30T06:24:35.970 INFO:teuthology.orchestra.run.smithi079.stdout: "tiebreaker_mon": "", 2023-10-30T06:24:35.970 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_ranks: ": "", 2023-10-30T06:24:35.970 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "persistent": [ 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:35.971 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:35.972 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:35.972 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:35.972 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:35.972 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:35.972 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: "optional": [] 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: "mons": [ 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.973 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 0, 2023-10-30T06:24:35.974 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "a", 2023-10-30T06:24:35.974 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:35.974 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:35.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.974 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3300", 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:35.975 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789", 2023-10-30T06:24:35.976 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.976 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.976 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.976 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789/0", 2023-10-30T06:24:35.977 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6789/0", 2023-10-30T06:24:35.977 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:35.977 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:35.977 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:35.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.978 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 1, 2023-10-30T06:24:35.978 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "b", 2023-10-30T06:24:35.978 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:35.978 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:3300", 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.980 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:35.980 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789", 2023-10-30T06:24:35.980 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.980 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.981 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789/0", 2023-10-30T06:24:35.981 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6789/0", 2023-10-30T06:24:35.981 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:35.981 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:35.982 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:35.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.982 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 2, 2023-10-30T06:24:35.982 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "c", 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3301", 2023-10-30T06:24:35.983 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.984 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:35.984 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790", 2023-10-30T06:24:35.984 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:35.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.985 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.985 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790/0", 2023-10-30T06:24:35.985 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6790/0", 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.987 INFO:teuthology.orchestra.run.smithi079.stdout: "feature_map": { 2023-10-30T06:24:35.987 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": [ 2023-10-30T06:24:35.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.987 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:35.987 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:35.988 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 1 2023-10-30T06:24:35.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.988 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:35.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client": [ 2023-10-30T06:24:35.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.989 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:35.989 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:35.989 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 3 2023-10-30T06:24:35.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.989 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:35.990 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": [ 2023-10-30T06:24:35.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:35.990 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:35.990 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:35.990 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 1 2023-10-30T06:24:35.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:35.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:35.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:35.991 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false 2023-10-30T06:24:35.991 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:35.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:35.994 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:35.991+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:35.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2023-10-30T06:24:36.094 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:36.093+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:36.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:24:36.168 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:36.169 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:36.175 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:36.175 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "c", 2023-10-30T06:24:36.175 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 2, 2023-10-30T06:24:36.176 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "peon", 2023-10-30T06:24:36.176 INFO:teuthology.orchestra.run.smithi079.stdout: "election_epoch": 8, 2023-10-30T06:24:36.177 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum": [ 2023-10-30T06:24:36.177 INFO:teuthology.orchestra.run.smithi079.stdout: 0, 2023-10-30T06:24:36.177 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:24:36.177 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:24:36.177 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.178 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_age": 73, 2023-10-30T06:24:36.178 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:36.178 INFO:teuthology.orchestra.run.smithi079.stdout: "required_con": "2449958758054445060", 2023-10-30T06:24:36.178 INFO:teuthology.orchestra.run.smithi079.stdout: "required_mon": [ 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.179 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.180 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.180 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.180 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.180 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.180 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.181 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_con": "4540138322906710015", 2023-10-30T06:24:36.181 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_mon": [ 2023-10-30T06:24:36.181 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.181 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.182 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.182 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.182 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.182 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.182 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.183 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.183 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.183 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.183 INFO:teuthology.orchestra.run.smithi079.stdout: "outside_quorum": [], 2023-10-30T06:24:36.184 INFO:teuthology.orchestra.run.smithi079.stdout: "extra_probe_peers": [], 2023-10-30T06:24:36.184 INFO:teuthology.orchestra.run.smithi079.stdout: "sync_provider": [], 2023-10-30T06:24:36.184 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": { 2023-10-30T06:24:36.184 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 1, 2023-10-30T06:24:36.184 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:36.185 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:36.185 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:36.185 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release": 18, 2023-10-30T06:24:36.185 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release_name": "reef", 2023-10-30T06:24:36.185 INFO:teuthology.orchestra.run.smithi079.stdout: "election_strategy": 3, 2023-10-30T06:24:36.186 INFO:teuthology.orchestra.run.smithi079.stdout: "disallowed_leaders: ": "", 2023-10-30T06:24:36.186 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false, 2023-10-30T06:24:36.186 INFO:teuthology.orchestra.run.smithi079.stdout: "tiebreaker_mon": "", 2023-10-30T06:24:36.186 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_ranks: ": "", 2023-10-30T06:24:36.186 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:36.187 INFO:teuthology.orchestra.run.smithi079.stdout: "persistent": [ 2023-10-30T06:24:36.187 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.187 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.187 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.187 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.188 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.188 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.188 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.188 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.188 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.189 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.189 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.189 INFO:teuthology.orchestra.run.smithi079.stdout: "optional": [] 2023-10-30T06:24:36.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.189 INFO:teuthology.orchestra.run.smithi079.stdout: "mons": [ 2023-10-30T06:24:36.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.190 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 0, 2023-10-30T06:24:36.190 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "a", 2023-10-30T06:24:36.190 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.190 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.191 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3300", 2023-10-30T06:24:36.191 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.192 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789", 2023-10-30T06:24:36.192 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.193 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789/0", 2023-10-30T06:24:36.194 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6789/0", 2023-10-30T06:24:36.194 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.194 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.194 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.195 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 1, 2023-10-30T06:24:36.196 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "b", 2023-10-30T06:24:36.196 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.196 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.197 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:3300", 2023-10-30T06:24:36.197 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.198 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.198 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789", 2023-10-30T06:24:36.198 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.199 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.199 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789/0", 2023-10-30T06:24:36.200 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6789/0", 2023-10-30T06:24:36.200 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.200 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.201 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.202 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 2, 2023-10-30T06:24:36.202 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "c", 2023-10-30T06:24:36.203 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.203 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.204 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3301", 2023-10-30T06:24:36.204 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.205 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790", 2023-10-30T06:24:36.205 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.206 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790/0", 2023-10-30T06:24:36.206 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6790/0", 2023-10-30T06:24:36.207 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.207 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.207 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.208 INFO:teuthology.orchestra.run.smithi079.stdout: "feature_map": { 2023-10-30T06:24:36.208 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": [ 2023-10-30T06:24:36.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.208 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.209 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.209 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 1 2023-10-30T06:24:36.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.209 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.209 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": [ 2023-10-30T06:24:36.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.210 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.210 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.210 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 3 2023-10-30T06:24:36.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.211 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client": [ 2023-10-30T06:24:36.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.212 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.212 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.212 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 2 2023-10-30T06:24:36.213 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.213 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.213 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false 2023-10-30T06:24:36.213 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:36.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:36.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2023-10-30T06:24:36.217 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:36.214+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:36.316 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:36.314+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:36.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:24:36.381 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:24:36.381 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.396 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:36.396 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:36.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:36.428 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:36.428 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "b", 2023-10-30T06:24:36.428 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 1, 2023-10-30T06:24:36.429 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "peon", 2023-10-30T06:24:36.429 INFO:teuthology.orchestra.run.smithi079.stdout: "election_epoch": 8, 2023-10-30T06:24:36.429 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum": [ 2023-10-30T06:24:36.429 INFO:teuthology.orchestra.run.smithi079.stdout: 0, 2023-10-30T06:24:36.429 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:24:36.430 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:24:36.430 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.430 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_age": 73, 2023-10-30T06:24:36.430 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:36.430 INFO:teuthology.orchestra.run.smithi079.stdout: "required_con": "2449958758054445060", 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "required_mon": [ 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.431 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.432 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.432 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.432 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.432 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.433 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.433 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.433 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_con": "4540138322906710015", 2023-10-30T06:24:36.433 INFO:teuthology.orchestra.run.smithi079.stdout: "quorum_mon": [ 2023-10-30T06:24:36.434 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.434 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.434 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.434 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.434 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.435 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.435 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.435 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.435 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.435 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: "outside_quorum": [], 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: "extra_probe_peers": [], 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: "sync_provider": [], 2023-10-30T06:24:36.436 INFO:teuthology.orchestra.run.smithi079.stdout: "monmap": { 2023-10-30T06:24:36.437 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 1, 2023-10-30T06:24:36.437 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:36.437 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:36.437 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:22:31.747585Z", 2023-10-30T06:24:36.437 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release": 18, 2023-10-30T06:24:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: "min_mon_release_name": "reef", 2023-10-30T06:24:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: "election_strategy": 3, 2023-10-30T06:24:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: "disallowed_leaders: ": "", 2023-10-30T06:24:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false, 2023-10-30T06:24:36.438 INFO:teuthology.orchestra.run.smithi079.stdout: "tiebreaker_mon": "", 2023-10-30T06:24:36.439 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_ranks: ": "", 2023-10-30T06:24:36.439 INFO:teuthology.orchestra.run.smithi079.stdout: "features": { 2023-10-30T06:24:36.439 INFO:teuthology.orchestra.run.smithi079.stdout: "persistent": [ 2023-10-30T06:24:36.439 INFO:teuthology.orchestra.run.smithi079.stdout: "kraken", 2023-10-30T06:24:36.439 INFO:teuthology.orchestra.run.smithi079.stdout: "luminous", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "mimic", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap-prune", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "nautilus", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "octopus", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pacific", 2023-10-30T06:24:36.440 INFO:teuthology.orchestra.run.smithi079.stdout: "elector-pinging", 2023-10-30T06:24:36.441 INFO:teuthology.orchestra.run.smithi079.stdout: "quincy", 2023-10-30T06:24:36.441 INFO:teuthology.orchestra.run.smithi079.stdout: "reef" 2023-10-30T06:24:36.441 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.441 INFO:teuthology.orchestra.run.smithi079.stdout: "optional": [] 2023-10-30T06:24:36.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.442 INFO:teuthology.orchestra.run.smithi079.stdout: "mons": [ 2023-10-30T06:24:36.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.442 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 0, 2023-10-30T06:24:36.442 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "a", 2023-10-30T06:24:36.442 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.443 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.443 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3300", 2023-10-30T06:24:36.443 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.444 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789", 2023-10-30T06:24:36.444 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6789/0", 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6789/0", 2023-10-30T06:24:36.445 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.446 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.446 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.448 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 1, 2023-10-30T06:24:36.448 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "b", 2023-10-30T06:24:36.448 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.448 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.449 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:3300", 2023-10-30T06:24:36.449 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.450 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789", 2023-10-30T06:24:36.450 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.450 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.450 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.451 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6789/0", 2023-10-30T06:24:36.451 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6789/0", 2023-10-30T06:24:36.451 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.451 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: "rank": 2, 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "c", 2023-10-30T06:24:36.452 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:36.453 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:36.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.453 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:36.453 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:3301", 2023-10-30T06:24:36.454 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:36.455 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790", 2023-10-30T06:24:36.455 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 0 2023-10-30T06:24:36.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.456 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.457 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6790/0", 2023-10-30T06:24:36.457 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6790/0", 2023-10-30T06:24:36.457 INFO:teuthology.orchestra.run.smithi079.stdout: "priority": 0, 2023-10-30T06:24:36.458 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:24:36.458 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_location": "{}" 2023-10-30T06:24:36.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.459 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.459 INFO:teuthology.orchestra.run.smithi079.stdout: "feature_map": { 2023-10-30T06:24:36.460 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": [ 2023-10-30T06:24:36.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.460 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.461 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.461 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 1 2023-10-30T06:24:36.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.461 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.462 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": [ 2023-10-30T06:24:36.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.462 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.463 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.463 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 5 2023-10-30T06:24:36.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.464 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": [ 2023-10-30T06:24:36.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.464 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 3 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": [ 2023-10-30T06:24:36.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:36.466 INFO:teuthology.orchestra.run.smithi079.stdout: "features": "0x3f01cfbffffdffff", 2023-10-30T06:24:36.466 INFO:teuthology.orchestra.run.smithi079.stdout: "release": "luminous", 2023-10-30T06:24:36.466 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 1 2023-10-30T06:24:36.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:36.467 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:36.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:36.467 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": false 2023-10-30T06:24:36.467 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:36.468 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:36.445+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:36.469 INFO:tasks.mon_recovery:restarting each monitor in turn 2023-10-30T06:24:36.470 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T06:24:36.471 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:24:36.476 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:24:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.492 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:24:36.488+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 85989) UID: 0 2023-10-30T06:24:36.493 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:24:36.489+0000 9f64640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-30T06:24:36.518 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:36.517+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:36.581 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:24:36.582 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.598 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:36.598 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:36.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:36.619 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:36.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:36.688 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:24:36.689 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.720 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:36.719+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:36.738 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:36.738 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:36.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:36.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:24:36.781 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.832 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:36.830+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:36.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:24:36.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:24:36.922 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:36.921+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:36.986 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:24:36.986 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:24:37.023 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:37.022+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:37.032 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:37.033 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:37.042 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:37.091 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:24:37.092 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:24:37.145 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:37.144+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:37.216 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:24:37.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:24:37.217 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:24:37.218 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:24:37.218 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:24:37.225 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:37.223+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:37.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:37.234 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:32.435 86020== 2023-10-30T06:24:37.234 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:32.435 86020== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:24:37.244 INFO:tasks.ceph.mon.a.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:24:37.325 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:37.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:37.394 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:24:37.395 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:24:37.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:24:37.395 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:24:37.395 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:24:37.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:37.448 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:37.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:37.527 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:37.526+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:37.530 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:37.531 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:37.531 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:37.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:37.628 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:37.627+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:37.711 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:37.711 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:37.712 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:37.712 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:37.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:37.728 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:37.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:37.832 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:37.832+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:37.848 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:37.849 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:37.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:37.849 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:24:37.849 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:37.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:37.931 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:37.929+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:38.027 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:38.028 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:38.032 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:38.031+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:38.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:38.132 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:38.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:38.166 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:38.167 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:38.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:38.233 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:38.233+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:38.334 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:38.334+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:38.376 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:38.377 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:38.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:38.377 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:24:38.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.378 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:24:38.378 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:38.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 25.022963664999999, 2023-10-30T06:24:38.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:24:38.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:24:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:24:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:24:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:24:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:24:38.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:24:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:24:38.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:24:38.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:24:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:24:38.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:24:38.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:24:38.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:24:38.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:24:38.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:24:38.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:24:38.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:24:38.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:24:38.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:24:38.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:24:38.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:24:38.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:24:38.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:24:38.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:24:38.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:24:38.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:24:38.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:24:38.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:24:38.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.701808474, 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:24:38.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:24:38.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:38.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:38.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:24:38.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:24:38.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:24:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:24:38.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:24:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:24:38.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:24:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:24:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:24:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:24:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:24:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:24:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:24:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:24:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:24:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:24:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:38.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.697563917, 2023-10-30T06:24:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:24:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:24:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:24:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:24:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:24:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:24:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:24:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:24:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:24:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:24:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:24:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:24:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:24:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:24:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:24:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:24:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:24:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:24:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:24:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:24:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.612105492000001, 2023-10-30T06:24:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:24:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:24:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:24:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:24:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:24:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:24:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:24:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:24:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:24:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:24:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:24:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:24:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:24:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:24:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:24:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:24:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:24:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:24:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:24:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:24:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:24:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:24:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.342862276000002, 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:24:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:24:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:24:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:24:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:24:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:24:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:24:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:24:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:24:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:24:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:24:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:24:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:24:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:24:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:24:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:24:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.342030865000002, 2023-10-30T06:24:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:24:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:24:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:24:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:24:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:24:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:24:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:24:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:24:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:24:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:24:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:24:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:24:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:24:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:24:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:24:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:24:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:24:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:24:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:24:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:24:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.341363737999998, 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:24:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:24:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:24:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:24:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:24:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:24:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:24:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:24:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:24:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:24:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:24:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:24:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:24:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:24:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:24:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:24:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:24:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:24:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:24:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.319089667, 2023-10-30T06:24:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:24:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:24:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:24:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:24:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:24:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:24:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:24:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:24:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:24:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:24:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:24:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:24:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:24:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:24:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:24:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.318441811, 2023-10-30T06:24:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:24:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:24:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:24:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:24:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:24:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:24:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:24:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:24:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:24:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:24:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:24:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:24:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:24:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:24:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:24:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:24:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 22.317755737999999, 2023-10-30T06:24:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:24:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:24:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:24:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:24:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:24:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:24:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:24:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:24:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:24:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:24:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:24:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:24:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:24:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:24:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:24:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.246873770000001, 2023-10-30T06:24:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:24:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:24:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:24:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:24:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:24:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:24:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:24:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:24:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:24:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:24:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:24:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:24:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:24:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:24:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:24:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:24:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.246209803999999, 2023-10-30T06:24:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:24:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:24:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:24:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:24:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:24:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:24:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:24:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:24:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:24:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:24:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:24:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:24:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:24:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:24:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:24:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:24:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:24:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 21.217755995000001, 2023-10-30T06:24:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:24:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:24:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:24:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:24:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:24:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:24:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:24:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:24:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:24:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:24:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:24:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:24:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:24:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:24:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:24:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:24:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:24:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:24:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:24:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:24:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:24:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.875989501999999, 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:24:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:24:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:24:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:24:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:24:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:24:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:24:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:24:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:24:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:24:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:24:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:24:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:24:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:24:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:24:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:24:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:24:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:24:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:24:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.875240831999999, 2023-10-30T06:24:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:24:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:24:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:24:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:24:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:24:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:24:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:24:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:24:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:24:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:24:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:24:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:24:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:24:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:24:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:24:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:24:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.874572449, 2023-10-30T06:24:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:24:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:24:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:24:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:24:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:24:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:24:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:24:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:24:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:24:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:24:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:24:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:24:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:24:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:24:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:24:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:38.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.864721775, 2023-10-30T06:24:38.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:24:38.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:24:38.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.638 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:38.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:38.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:24:38.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:24:38.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:24:38.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:24:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:24:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:24:38.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:24:38.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:24:38.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:24:38.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:24:38.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:24:38.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:24:38.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:24:38.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:24:38.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:24:38.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:38.652 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.864047960000001, 2023-10-30T06:24:38.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:24:38.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:24:38.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.653 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:38.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:24:38.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:24:38.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:24:38.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:24:38.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:24:38.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:24:38.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:24:38.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:24:38.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:24:38.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:24:38.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:24:38.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:24:38.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:24:38.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:24:38.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:24:38.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:38.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.863399576999999, 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:24:38.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:38.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:24:38.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:24:38.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:24:38.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:24:38.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:24:38.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:24:38.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:24:38.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:24:38.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:24:38.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:24:38.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:24:38.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:24:38.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:24:38.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:24:38.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:24:38.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:24:38.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:24:38.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:24:38.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:38.683 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:38.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 13.859283111, 2023-10-30T06:24:38.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:24:38.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:38.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:38.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:38.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:38.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:38.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:24:38.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:38.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:38.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:38.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:38.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:24:38.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:24:38.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:38.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:24:38.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:24:38.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:38.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:24:38.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:24:38.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:38.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:24:38.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:24:38.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:38.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:24:38.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:24:38.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:38.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:24:38.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:24:38.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:38.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:24:38.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:24:38.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:38.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:24:38.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:24:38.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:38.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:38.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:38.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:38.698 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:38.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:38.700 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:38.698+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:38.760 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:24:38.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:24:38.761 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2023-10-30T06:24:38.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:24:38.802 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:38.801+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:38.891 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:38.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:38.891 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:38.891 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:38.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:38.903 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:38.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:39.029 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:39.028+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:39.045 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:39.045 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:39.046 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:39.046 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:39.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:39.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:24:39.105 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:39.104+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:39.206 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:39.205+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:39.230 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:39.231 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:39.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:39.231 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:24:39.231 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:39.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:39.324 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:39.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:39.384 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:24:39.384 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.412 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:39.411+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:39.428 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:39.428 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:39.428 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:39.428 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:39.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:39.452 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 25, 2023-10-30T06:24:39.452 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:24:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:24:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:24:34.267172+0000", 2023-10-30T06:24:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:24:39.453 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:24:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:24:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5801984, 2023-10-30T06:24:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:24:39.454 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2023-10-30T06:24:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:24:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:24:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:24:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:24:39.455 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:24:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:24:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:24:39.456 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:24:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:24:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:24:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:24:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:24:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "jewel", 2023-10-30T06:24:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:24:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:24:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:24:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:24:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:24:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:24:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:24:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:24:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:24:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:24:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:24:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:24:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:24:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:24:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:24:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:24:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:24:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:24:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:24:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:24:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:24:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:24:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:24:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:24:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:24:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:24:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:24:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:24:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:24:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:24:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:24:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:24:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:24:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:24:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:24:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:24:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:24:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:24:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:24:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:24:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:24:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:24:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:24:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:24:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:24:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:24:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:24:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.469 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:24:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:24:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:24:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:24:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:24:39.470 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:24:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:24:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:24:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:24:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:24:39.471 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:24:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:24:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:24:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:24:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.472 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:24:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:24:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:24:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 8, 2023-10-30T06:24:39.473 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 8, 2023-10-30T06:24:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.25, 2023-10-30T06:24:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:24:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:24:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2023-10-30T06:24:39.474 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:24:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2023-10-30T06:24:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.475 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:24:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:24:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:24:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:24:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:24:39.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:24:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:24:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:24:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:24:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:24:39.477 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:24:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:24:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:24:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:24:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:24:39.478 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:24:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:24:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:24:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:24:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:24:39.479 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:24:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:24:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:24:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:24:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:24:39.480 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:24:39.481 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:24:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:24:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:24:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:24:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:24:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:24:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:24:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:24:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:24:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:24:39.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:24:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:24:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:24:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:24:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:24:39.484 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:24:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:24:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:24:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:24:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:24:39.485 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:24:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:24:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:24:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:24:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.486 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:24:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:24:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:24:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:24:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:24:39.487 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:24:39.488 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:24:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:24:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:24:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:24:39.489 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 4.9800000190734863, 2023-10-30T06:24:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 4.9800000190734863, 2023-10-30T06:24:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.87999999523162842, 2023-10-30T06:24:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 4.380000114440918, 2023-10-30T06:24:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 4.380000114440918, 2023-10-30T06:24:39.490 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 1, 2023-10-30T06:24:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:24:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 1 2023-10-30T06:24:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.491 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:24:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:24:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:24:39.492 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.493 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.494 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:24:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.496 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:24:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.501 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:24:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.502 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:24:39.503 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:24:39.504 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:24:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:24:39.505 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:24:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:24:39.507 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.508 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:24:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 18, 2023-10-30T06:24:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.509 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:24:39.510 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:24:39.511 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.512 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:24:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:24:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:24:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.518 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:24:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.519 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:24:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:24:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.520 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:24:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:24:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:24:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:24:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.521 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.523 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:39.524 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:24:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:24:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:24:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.531 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:24:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.532 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:24:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:24:39.534 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:24:39.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:24:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:24:39.536 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:24:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:24:39.537 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:24:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:24:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.539 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:39.540 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:24:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:24:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:24:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:24:39.547 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:24:39.548 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.549 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:24:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:24:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:24:39.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:24:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:24:39.552 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:24:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:24:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:24:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:24:39.554 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.555 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.556 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:24:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:24:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:24:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.561 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:24:39.561 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.562 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.562 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.563 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:24:39.563 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.564 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:24:39.564 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.565 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.565 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.566 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:24:39.566 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.567 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:24:39.567 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:24:39.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:24:39.568 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.569 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.569 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.570 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.571 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.571 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.571 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.571 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:39.571 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:39.572 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.572 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.572 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.572 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.573 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:24:39.573 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.574 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:24:39.574 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.574 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.575 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.575 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.576 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:24:39.577 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.577 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.578 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.578 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:24:39.578 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.579 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:24:39.579 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.580 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.580 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:24:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:24:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:24:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:24:39.584 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.584 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.584 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.585 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:24:39.585 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:24:39.585 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.585 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.586 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.586 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.586 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.586 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.586 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:24:39.587 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:24:39.587 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.587 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.587 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.587 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:24:39.588 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.590 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.590 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.591 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:24:39.591 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.592 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.593 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.593 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:24:39.593 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.594 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:24:39.594 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.595 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.595 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.597 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:24:39.597 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:24:39.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.598 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:24:39.598 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:24:39.598 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:24:39.598 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:24:39.599 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.599 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.599 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:24:39.600 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:24:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:24:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:24:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:24:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:24:39.601 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:24:39.602 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:24:39.602 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:24:39.602 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:24:39.602 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.604 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:24:39.604 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.605 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:24:39.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:24:39.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:24:39.607 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.608 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:24:39.608 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.609 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:24:39.610 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.610 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:24:39.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:24:39.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:24:39.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:24:39.611 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:24:39.612 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:24:39.612 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:24:39.613 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:24:39.614 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:24:39.614 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:24:39.614 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:24:39.614 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:24:39.614 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:39.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.615 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:39.615 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:24:39.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.615 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:24:39.616 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.616 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.616 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.616 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.616 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.617 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:24:39.617 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.617 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:24:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.618 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.619 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:24:39.619 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.619 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.620 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:24:39.621 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.621 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:24:39.621 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.622 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.622 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.622 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.622 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.622 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:24:39.623 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.623 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:24:39.623 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.624 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.624 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.624 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.624 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:24:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.625 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:24:39.626 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.626 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.626 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.626 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.626 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.627 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:24:39.627 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.627 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:24:39.628 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.628 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.628 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.628 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.628 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.629 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:24:39.629 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:39.629 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:24:39.630 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:24:39.630 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:24:39.630 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:24:39.630 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:24:39.630 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:24:39.631 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:24:39.631 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:24:39.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.631 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:24:39.631 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [], 2023-10-30T06:24:39.632 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2023-10-30T06:24:39.632 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:24:39.632 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:24:39.632 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:24:39.632 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:24:39.633 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:24:39.633 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:24:39.633 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:24:39.633 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:24:39.633 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:24:39.634 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:24:39.634 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:24:39.634 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:24:39.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:39.635 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:24:39.635 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:24:39.635 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:24:39.635 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:24:39.635 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:24:39.636 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:24:39.636 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:24:39.636 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:24:39.636 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:24:39.636 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:24:39.637 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:24:39.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:39.637 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:39.638 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-30T06:24:39.640 INFO:tasks.thrashosds.thrasher:j is {'epoch': 25, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:24:34.267172+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:24:39.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:24:39.641 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:24:39.641 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.641 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:24:39.642 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.658 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:24:39.659 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.659 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:39.658+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:39.739 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:39.738+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:39.746 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:24:39.746 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.769 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:39.769 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:39.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:39.840 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:39.839+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:39.849 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:24:39.850 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.906 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:39.907 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:39.916 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:39.938 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:24:39.939 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:24:39.959 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:39.958+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:40.032 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:24:40.032 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:24:40.042 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:40.041+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:40.143 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:40.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.228 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:40.242 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:24:40.243 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:24:40.253 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2023-10-30T06:24:39.261829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218200,"kb_used_data":2032,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756520,"statfs":{"total":773094113280,"available":772870676480,"internally_reserved":0,"allocated":2080768,"data_stored":1490508,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":64,"apply_latency_ms":64,"commit_latency_ns":64000000,"apply_latency_ns":64000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001866"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.621313+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:24:21.621313+0000","last_peered":"2023-10-30T06:24:21.621313+0000","last_clean":"2023-10-30T06:24:21.621313+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:24:21.621313+0000","last_undegraded":"2023-10-30T06:24:21.621313+0000","last_fullsized":"2023-10-30T06:24:21.621313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.348491+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:24:21.348491+0000","last_peered":"2023-10-30T06:24:21.348491+0000","last_clean":"2023-10-30T06:24:21.348491+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:24:21.348491+0000","last_undegraded":"2023-10-30T06:24:21.348491+0000","last_fullsized":"2023-10-30T06:24:21.348491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.620021+0000","last_change":"2023-10-30T06:24:20.087820+0000","last_active":"2023-10-30T06:24:21.620021+0000","last_peered":"2023-10-30T06:24:21.620021+0000","last_clean":"2023-10-30T06:24:21.620021+0000","last_became_active":"2023-10-30T06:24:20.060063+0000","last_became_peered":"2023-10-30T06:24:20.060063+0000","last_unstale":"2023-10-30T06:24:21.620021+0000","last_undegraded":"2023-10-30T06:24:21.620021+0000","last_fullsized":"2023-10-30T06:24:21.620021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:22:25.280973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.464682+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:24:21.464682+0000","last_peered":"2023-10-30T06:24:21.464682+0000","last_clean":"2023-10-30T06:24:21.464682+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:24:21.464682+0000","last_undegraded":"2023-10-30T06:24:21.464682+0000","last_fullsized":"2023-10-30T06:24:21.464682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.616753+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:24:21.616753+0000","last_peered":"2023-10-30T06:24:21.616753+0000","last_clean":"2023-10-30T06:24:21.616753+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:24:21.616753+0000","last_undegraded":"2023-10-30T06:24:21.616753+0000","last_fullsized":"2023-10-30T06:24:21.616753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.472975+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:24:21.472975+0000","last_peered":"2023-10-30T06:24:21.472975+0000","last_clean":"2023-10-30T06:24:21.472975+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:24:21.472975+0000","last_undegraded":"2023-10-30T06:24:21.472975+0000","last_fullsized":"2023-10-30T06:24:21.472975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.612153+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:24:21.612153+0000","last_peered":"2023-10-30T06:24:21.612153+0000","last_clean":"2023-10-30T06:24:21.612153+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:24:21.612153+0000","last_undegraded":"2023-10-30T06:24:21.612153+0000","last_fullsized":"2023-10-30T06:24:21.612153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:21.618390+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:24:21.618390+0000","last_peered":"2023-10-30T06:24:21.618390+0000","last_clean":"2023-10-30T06:24:21.618390+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:24:21.618390+0000","last_undegraded":"2023-10-30T06:24:21.618390+0000","last_fullsized":"2023-10-30T06:24:21.618390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-30T06:24:24.703877+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:24:24.703877+0000","last_peered":"2023-10-30T06:24:24.703877+0000","last_clean":"2023-10-30T06:24:24.703877+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:24:24.703877+0000","last_undegraded":"2023-10-30T06:24:24.703877+0000","last_fullsized":"2023-10-30T06:24:24.703877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411337,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":486046,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2669999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.923999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.561}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.776}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.09}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.688000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":88206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.125}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0009999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.922000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.6679999999999993}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.893000000000001}]}]},{"osd":3,"up_from":16,"seq":68719476746,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":88206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.466000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.963999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.123999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.472999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.158000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":26.582000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":30.222999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476744,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":88206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2290000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9179999999999993}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.966999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.372}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.064}]}]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":88206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.798999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4139999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.407999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.246}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.972}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.789000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.628}]}]},{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":82796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9450000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.823}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4130000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.148999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.246}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.341000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":480636,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.845000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.109}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.3730000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0559999999999992}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.269}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.347000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":88206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.3680000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.885999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.499000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":29.881}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.148}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.574999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.327000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:24:40.255 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [2, 1] 2023-10-30T06:24:40.255 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '2', '1'] 2023-10-30T06:24:40.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 2 1 2023-10-30T06:24:40.285 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:40.283+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:40.345 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:40.344+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:40.362 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.362 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:40.447 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:40.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:40.543 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.544 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.547 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:40.546+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:40.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:40.650 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:40.649+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:40.727 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.728 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:40.751 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:40.750+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:40.853 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:40.851+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:40.903 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:40.903 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:40.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:40.954 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:40.952+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:41.054 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:41.053+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:41.079 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.079 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:41.155 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:41.154+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:41.255 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.256 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.256 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:41.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:41.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:41.357 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:41.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:41.391 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.391 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.400 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:41.458 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:41.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:41.558 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:41.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:41.566 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.566 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:41.659 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:41.658+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:41.701 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.702 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:41.796 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:41.794+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:41.864 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:41.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:41.877 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:41.878 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:41.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:41.962 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:41.961+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:42.016 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.017 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:42.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:24:42.062 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:42.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:42.158 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.158 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.163 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:42.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:42.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:42.264 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:42.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:42.336 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.336 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:42.364 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:42.363+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:42.466 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:42.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:42.513 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.513 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:42.567 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:42.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:42.573 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:24:42.574 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-10-30T06:24:42.574 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T06:24:42.574 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:24:42.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:24:42.654 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.654 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:42.667 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:42.665+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:42.768 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:42.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:42.790 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.791 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:42.885 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:42.883+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:42.969 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:42.969 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:42.970 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:42.969+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:42.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:43.071 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:43.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:43.149 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.149 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:43.172 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:43.170+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:43.273 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:43.271+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:43.327 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:43.379 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:43.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:43.475 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:43.473+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:43.503 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.503 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:43.576 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:43.575+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:43.677 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:43.676+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:43.683 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.683 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:43.778 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:43.776+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:43.820 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.820 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:43.878 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:43.877+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:43.966 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:43.967 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:43.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:43.979 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:43.977+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:44.080 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:44.079+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:44.141 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.141 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:44.191 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:44.190+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:44.283 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:44.281+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:44.316 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.316 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:44.384 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:44.382+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:44.451 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.452 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:44.501 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:44.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:44.586 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:44.585+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:44.631 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.631 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:44.686 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:44.685+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:44.766 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.766 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:44.820 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:44.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:44.888 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:44.887+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:44.903 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:44.903 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:44.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:44.989 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:44.988+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:45.077 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.077 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:45.090 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:45.089+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:45.211 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:45.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:45.225 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.225 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:45.292 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:45.291+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:45.393 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:45.391+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:45.402 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.403 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.412 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:45.494 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:45.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:45.537 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.538 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:45.595 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:45.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:45.696 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:45.694+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:45.724 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.724 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:45.796 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:45.795+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:45.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-30T06:24:45.871 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:45.872 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:45.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:45.898 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:45.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:46.001 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:45.999+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:46.055 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.055 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:46.109 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:46.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:46.194 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.194 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.201 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:46.200+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:46.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:46.210 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":83,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:24:46.302 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:46.301+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:46.335 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.335 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:46.402 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:46.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:46.470 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.471 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:46.503 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:46.502+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:46.604 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:46.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:46.654 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.655 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:46.706 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:46.703+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:46.806 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:46.805+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:46.834 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.834 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:46.907 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:46.906+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:46.973 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:46.974 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:46.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:47.028 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:47.026+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:47.108 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:47.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:47.114 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.115 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:47.210 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:47.208+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:47.291 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.292 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:47.310 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:47.309+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:47.411 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:47.409+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:47.472 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.472 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:47.522 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:47.520+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:47.613 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:47.612+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:47.645 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.646 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.654 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:47.714 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:47.713+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:47.785 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.786 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:47.838 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:47.837+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:47.916 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:47.914+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:47.920 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:47.921 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:47.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:48.019 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:48.017+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:48.097 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.097 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:48.148 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:48.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:48.222 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:48.221+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:48.275 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.275 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:48.323 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:48.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:48.411 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.411 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:48.423 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:48.422+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:48.524 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:48.522+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:48.544 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.545 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:48.625 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:48.624+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:48.680 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.681 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:48.726 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:48.725+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:48.817 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.818 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:48.827 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:48.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:48.928 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:48.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:48.954 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:48.954 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:48.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:49.029 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:49.028+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:49.090 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.090 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:49.141 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:49.140+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:49.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:24:49.265 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.266 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.270 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:49.268+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:49.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:49.330 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:49.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:49.401 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.401 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:49.452 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:49.451+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:49.532 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:49.532+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:49.537 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:49.633 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:49.632+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:49.673 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.673 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:49.734 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:49.732+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:49.809 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.809 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:49.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:49.835 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:49.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:49.936 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:49.935+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:49.992 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:49.992 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:50.046 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:50.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:50.128 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:50.128 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:50.138 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:50.136+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:50.238 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:50.237+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:50.303 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:50.304 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:50.339 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:50.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:50.440 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:50.439+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:50.479 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:50.480 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:50.574 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:50.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:50.643 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:50.642+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:50.659 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:50.659 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:50.753 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:50.751+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:50.835 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:50.835 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:50.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:50.844 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:50.842+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:50.944 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:50.943+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:51.012 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.012 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:51.062 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:51.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:51.146 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:51.146+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:51.197 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.197 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:51.247 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:51.246+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:51.331 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.331 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:51.347 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:51.346+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:51.448 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:51.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:51.476 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.477 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:51.569 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:51.568+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:51.651 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:51.649+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:51.651 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.652 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:51.751 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:51.750+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:51.830 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:51.831 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:51.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:51.852 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:51.851+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:51.953 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:51.952+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:52.004 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.005 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:52.054 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:52.052+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:52.145 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.145 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:52.156 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:52.154+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:52.257 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:52.255+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:52.283 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.283 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:52.357 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:52.356+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:52.418 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.418 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:52.468 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:52.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:52.559 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:52.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:52.593 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.594 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:52.661 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:52.659+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.732 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:52.787 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:52.786+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:52.862 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:52.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:52.869 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:52.869 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:52.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:52.963 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:52.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.044 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:53.064 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:53.062+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:53.165 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:53.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:53.180 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.181 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:53.267 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:53.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:53.324 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.325 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:53.378 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:53.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:53.461 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.462 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.468 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:53.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:53.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:53.570 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:53.569+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:53.639 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.639 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:53.671 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:53.670+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:53.772 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:53.770+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:53.818 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.819 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:53.873 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:53.872+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:53.954 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:53.954 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:53.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:54.008 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:54.006+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:54.077 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:54.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:54.090 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.090 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:54.184 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:54.183+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:54.274 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.274 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.277 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:54.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:54.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:54.377 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:54.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:54.452 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.452 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:54.478 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:54.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:54.578 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:54.577+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:54.628 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.628 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:54.679 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:54.678+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:54.780 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:54.779+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:54.803 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.803 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:54.882 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:54.880+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:54.937 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:54.937 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:54.946 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:54.994 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:54.990+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:55.073 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.074 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:55.084 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:55.082+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:55.184 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:55.182+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:55.222 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.222 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:55.285 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:55.284+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:55.359 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.359 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:24:55.410 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:55.408+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:55.486 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:55.485+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:55.533 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.534 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:24:55.587 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:55.586+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:55.689 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:55.687+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:55.709 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.710 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:24:55.790 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:55.788+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:55.853 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:55.853 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:55.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:24:55.913 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:55.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:56.002 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:55.999+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:56.044 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.045 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:24:56.095 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:56.093+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:56.179 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.179 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:24:56.198 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:56.196+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:56.295 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:56.294+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:56.322 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.322 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.330 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:24:56.399 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:56.397+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:56.499 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.499 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.502 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:56.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:56.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:24:56.598 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:56.596+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:56.635 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.635 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:24:56.699 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:56.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:56.777 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.778 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:24:56.800 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:56.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:56.900 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:56.899+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:56.920 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:56.921 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:56.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:24:57.002 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:57.000+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:57.065 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:57.066 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:57.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:24:57.102 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:57.101+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:57.203 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:57.203 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:57.204 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:57.202+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:57.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:24:57.304 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:57.302+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:57.379 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:57.380 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:57.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:24:57.404 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:57.403+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:57.505 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:57.504+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:57.572 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:24:57.573 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:24:57.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:24:57.624 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:24:57.624 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:24:57.625 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:57.623+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:57.712 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:24:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:24:57.713 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:57.711+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:57.757 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:57.758 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:57.758 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:57.758 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:57.758 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:57.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:24:57.815 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:24:57.816 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:24:57.817 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:57.814+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:57.863 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:24:57.918 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:57.916+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:57.926 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:24:57.927 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:24:57.973 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.5 pg_upmap mapping to [2,1] 2023-10-30T06:24:57.983 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:24:57.997 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:24:57.998 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T06:24:57.998 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-10-30T06:24:58.019 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:58.020 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:58.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:58.020 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:24:58.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.021 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:24:58.021 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:58.021 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 44.661981441999998, 2023-10-30T06:24:58.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:24:58.021 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.022 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.022 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:24:58.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.023 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:58.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:24:58.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:24:58.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:24:58.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:24:58.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:24:58.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:24:58.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:24:58.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:24:58.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:24:58.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:24:58.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:24:58.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:24:58.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:24:58.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:24:58.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:24:58.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:24:58.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:24:58.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:24:58.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:24:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:24:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:24:58.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:24:58.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:24:58.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:24:58.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:24:58.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:24:58.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.340826251000003, 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:24:58.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.040 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:24:58.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.041 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:58.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:24:58.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:24:58.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:24:58.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:24:58.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:24:58.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:24:58.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:24:58.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:24:58.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:24:58.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:24:58.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:24:58.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:24:58.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:24:58.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:24:58.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:24:58.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:24:58.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:24:58.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:58.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.336581694000003, 2023-10-30T06:24:58.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:24:58.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.055 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:24:58.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:24:58.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:24:58.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:24:58.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:24:58.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:24:58.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:24:58.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:24:58.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:24:58.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:24:58.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:24:58.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:24:58.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:24:58.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.069 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.069 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:58.069 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 42.251123268999997, 2023-10-30T06:24:58.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:24:58.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:24:58.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.071 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:24:58.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:24:58.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:24:58.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:24:58.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:24:58.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:24:58.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:24:58.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:24:58.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:24:58.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:24:58.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:24:58.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:24:58.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:24:58.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:24:58.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:24:58.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:24:58.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:24:58.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:24:58.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:24:58.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:58.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.981880052999998, 2023-10-30T06:24:58.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:24:58.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:24:58.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:24:58.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:24:58.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:24:58.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:24:58.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:24:58.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:24:58.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:24:58.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:24:58.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:24:58.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:24:58.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:24:58.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:24:58.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:24:58.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:24:58.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:24:58.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:24:58.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:24:58.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:24:58.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:24:58.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:24:58.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:24:58.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.101 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:58.101 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.981048641999998, 2023-10-30T06:24:58.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:24:58.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:24:58.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:58.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:24:58.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:24:58.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:24:58.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:24:58.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:24:58.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:24:58.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:24:58.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:24:58.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:24:58.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:24:58.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:24:58.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:24:58.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:24:58.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:24:58.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:24:58.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:24:58.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:24:58.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:24:58.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.980381514999998, 2023-10-30T06:24:58.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:24:58.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.118 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:24:58.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:24:58.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:24:58.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:24:58.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:24:58.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:24:58.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:24:58.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:24:58.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:24:58.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:24:58.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:24:58.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:24:58.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:24:58.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:24:58.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:24:58.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:24:58.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:24:58.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:24:58.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:24:58.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:24:58.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.131 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.131 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.131 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.958107443999999, 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:58.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:24:58.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:24:58.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:24:58.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:24:58.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:24:58.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:24:58.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:24:58.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:24:58.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:24:58.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:24:58.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:24:58.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:24:58.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:24:58.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:24:58.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:24:58.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:24:58.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:24:58.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.147 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.147 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:58.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.957459587999999, 2023-10-30T06:24:58.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:24:58.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.148 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:24:58.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:58.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:24:58.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:24:58.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:24:58.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:24:58.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:24:58.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:24:58.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:24:58.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:24:58.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:24:58.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:24:58.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:24:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:24:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:24:58.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:24:58.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:24:58.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:24:58.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:24:58.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:58.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 41.956773515000002, 2023-10-30T06:24:58.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:24:58.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.163 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:24:58.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:58.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:24:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:24:58.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:24:58.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:24:58.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:24:58.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:24:58.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:24:58.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:24:58.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:24:58.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:24:58.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:24:58.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:24:58.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:24:58.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:24:58.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:24:58.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:24:58.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:24:58.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:24:58.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:24:58.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.177 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.178 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:58.178 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:58.178 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.885891547, 2023-10-30T06:24:58.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:24:58.179 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.179 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.180 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.180 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:24:58.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:58.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:24:58.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:24:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:24:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:24:58.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:24:58.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:24:58.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:24:58.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:24:58.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:24:58.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:24:58.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:24:58.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:24:58.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:24:58.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:24:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:24:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:24:58.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:24:58.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.194 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:58.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:58.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.885227581000002, 2023-10-30T06:24:58.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:24:58.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:58.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:24:58.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:24:58.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:24:58.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:24:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:24:58.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:24:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:24:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:24:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:24:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:24:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:24:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:24:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:24:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:24:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:24:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:24:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 40.856773771999997, 2023-10-30T06:24:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:24:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:24:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:24:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:24:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:24:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:24:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:24:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:24:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:24:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:24:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:24:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:24:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:24:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:24:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:24:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:24:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:24:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:24:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:24:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.515007279000002, 2023-10-30T06:24:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:24:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:24:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:24:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:24:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:24:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:24:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:24:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:24:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:24:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:24:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:24:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:24:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:24:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:24:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:24:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:24:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:24:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:24:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:24:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:24:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:24:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:24:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:24:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.514258609000002, 2023-10-30T06:24:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:24:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:24:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:24:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:24:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:24:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:24:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:24:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:24:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:24:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:24:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:24:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:24:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:24:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:24:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:24:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:24:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:24:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:24:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:24:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.513590225999998, 2023-10-30T06:24:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:24:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:24:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:24:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:24:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:24:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:24:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:24:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:24:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:24:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:24:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:24:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:24:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:24:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:24:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:24:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:24:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:24:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:24:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:24:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:24:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:24:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.503739551999999, 2023-10-30T06:24:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:24:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:24:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:24:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:24:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:24:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:24:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:24:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:24:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:24:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:24:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:24:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:24:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:24:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:24:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:24:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:24:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:24:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:24:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:24:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:24:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:24:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.503065737, 2023-10-30T06:24:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:24:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:24:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:24:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:24:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:24:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:24:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:24:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:24:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:24:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:24:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:24:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:24:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:24:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:24:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:24:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:24:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:24:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:24:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.502417354000002, 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:24:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:24:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:24:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:24:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:24:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:24:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:24:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:24:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:24:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:24:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:24:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:24:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:24:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:24:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:24:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:24:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:24:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:24:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:24:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:24:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:24:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:24:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.498300888000003, 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:24:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:24:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:24:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:24:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:24:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:24:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:24:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:24:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:24:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:24:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:24:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:24:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:24:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:24:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:24:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:24:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:24:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:24:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:24:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:24:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:24:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:24:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:24:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:24:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:24:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:24:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:24:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:24:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:24:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:24:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:24:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:24:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:24:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:24:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:24:58.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:24:58.334 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:58.335 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:58.017+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:58.335 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:24:58.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:24:58.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:24:58.337 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:24:58.338 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:24:58.338 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:24:58.339 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:24:58.339 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:24:58.340 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:24:58.341 INFO:tasks.ceph.mon.a:Started 2023-10-30T06:24:58.341 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:24:58.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:24:58.342 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:58.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:58.441 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:58.440+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:58.542 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:58.541+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:58.635 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:58.636 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:58.643 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:58.641+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:58.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:24:58.743 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:58.742+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:58.845 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:24:58.843+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:24:58.907 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:58.908 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:58.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:24:58.958 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:58.956+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:59.046 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:24:59.045+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:24:59.076 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:24:59.147 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:59.146+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:59.210 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:59.210 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:59.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:59.211 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:24:59.211 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:59.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:24:59.248 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:24:59.246+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:24:59.349 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:24:59.347+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:24:59.352 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:59.352 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:24:59.352 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:59.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:24:59.450 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:24:59.449+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:24:59.535 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:24:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:24:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:24:59.536 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:59.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:24:59.589 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:59.587+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:24:59.652 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:59.650+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:59.701 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:24:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:24:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:24:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:24:59.702 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:24:59.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:24:59.752 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:24:59.751+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:24:59.854 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:24:59.852+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:24:59.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:24:59.879 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:24:59.880 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:24:59.880 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:24:59.880 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:24:59.889 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:24:59.999 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:24:59.997+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:00.056 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:00.054+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:00.059 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:00.059 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:00.059 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:00.060 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:00.060 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:00.068 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:00.157 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:00.155+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:00.195 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:00.195 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:00.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:00.196 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:00.196 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:00.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:00.258 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:00.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:00.338 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:00.339 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:00.339 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:00.339 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:00.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:00.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:00.392 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:00.390+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:00.460 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:00.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:00.476 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:00.477 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:00.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:00.581 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:00.582 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:00.582 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:00.580+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:00.662 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:00.661+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:00.666 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:00.667 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:00.706 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:00.706 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:00.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:00.706 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:00.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.707 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:00.707 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:00.707 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.537832925000004, 2023-10-30T06:25:00.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:00.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:00.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:00.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:00.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:00.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:00.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.709 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:00.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:00.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:00.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:00.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:00.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:00.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:00.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:00.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:00.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:00.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:00.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.719 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.720 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:00.720 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:00.720 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 45.624659061999999, 2023-10-30T06:25:00.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:00.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:00.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:00.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:00.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:00.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:00.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:00.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:00.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:00.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:00.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:00.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:00.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.731 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.731 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.731 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.964481093000003, 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.732 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:00.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:00.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:00.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:00.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:00.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:00.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:00.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:00.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:00.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:00.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:00.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:00.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.742 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.889057886000003, 2023-10-30T06:25:00.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:00.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:00.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:00.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:00.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:00.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:00.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:00.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:00.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:00.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:00.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:00.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:00.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.754 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.754 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:00.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:00.755 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.755 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.755 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.617934755, 2023-10-30T06:25:00.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:00.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.757 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:00.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:00.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:00.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:00.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:00.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:00.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:00.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:00.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:00.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:00.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:00.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:00.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:00.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:00.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.767 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:00.768 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:00.768 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.587655044999998, 2023-10-30T06:25:00.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:00.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.769 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:00.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:00.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:00.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:00.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:00.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:00.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:00.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:00.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:00.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:00.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:00.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:00.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.779 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:00.779 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:00.779 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.213743960999999, 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:00.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:00.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:00.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:00.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:00.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:00.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:00.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:00.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:00.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:00.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:00.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:00.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.790 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.209245260000003, 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:00.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:00.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:00.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:00.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:00.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:00.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:00.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:00.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:00.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.802 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:00.802 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:00.802 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.917641320999998, 2023-10-30T06:25:00.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:00.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.803 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:00.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:00.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:00.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:00.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:00.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:00.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:00.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:00.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:00.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:00.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:00.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.812 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.813 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:00.813 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:00.814 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.911704004000001, 2023-10-30T06:25:00.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:00.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.814 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:00.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:00.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:00.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:00.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:00.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:00.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:00.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:00.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:00.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:00.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:00.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.824 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.910404989, 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.826 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:00.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:00.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:00.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:00.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:00.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:00.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:00.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:00.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:00.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:00.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:00.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.835 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.807416181000001, 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:00.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.837 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:00.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:00.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:00.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:00.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:00.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:00.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:00.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:00.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:00.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:00.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:00.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.846 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.847 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:00.847 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:00.847 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.535512914000002, 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:00.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:00.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:00.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:00.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:00.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:00.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:00.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:00.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:00.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:00.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:00.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:00.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:00.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.858 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.528014722000002, 2023-10-30T06:25:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.860 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:00.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:00.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:00.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:00.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:00.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:00.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:00.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:00.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:00.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:00.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:00.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.869 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.527353517000002, 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:00.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.871 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:00.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:00.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:00.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:00.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:00.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:00.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:00.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:00.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:00.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.880 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.514767679000002, 2023-10-30T06:25:00.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:00.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.882 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:00.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:00.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:00.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:00.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:00.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.886 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:00.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:00.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:00.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:00.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:00.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.892 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.892 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.892 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:00.893 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:00.893 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.508675466, 2023-10-30T06:25:00.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:00.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:00.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:00.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:00.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.898 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:00.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:00.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:00.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:00.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:00.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:00.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:00.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:00.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.904 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:00.904 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:00.904 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 43.507680125, 2023-10-30T06:25:00.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:00.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.905 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:00.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:00.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:00.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:00.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:00.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:00.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:00.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:00.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:00.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:00.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:00.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.914 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:00.915 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 36.185007554000002, 2023-10-30T06:25:00.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:00.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.916 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:00.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:00.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:00.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:00.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:00.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:00.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:00.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:00.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:00.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:00.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:00.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:00.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:00.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.926 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 36.151566068999998, 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:00.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:00.928 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:00.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:00.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:00.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:00.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:00.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:00.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:00.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:00.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:00.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:00.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:00.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:00.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:00.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:00.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:00.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:00.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:00.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:00.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:00.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.937 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:00.938 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:00.938 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:00.939 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:00.939 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:00.940 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:00.940 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:00.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:00.941 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:00.942 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:00.942 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:00.938+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:01.014 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:01.040 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:01.039+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:01.105 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:01.105 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:01.106 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.106 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:01.141 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:01.140+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:01.206 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:01.207 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:01.242 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:01.241+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:01.257 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.258 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:01.343 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:01.342+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:01.440 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.441 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.444 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:01.443+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:01.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:01.525 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:25:01.548 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:01.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:01.581 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.582 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:01.647 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:01.645+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:01.748 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:01.746+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:01.757 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.757 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:01.849 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:01.847+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:01.904 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:01.905 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:01.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:01.958 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:01.956+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:02.043 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.043 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.051 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:02.049+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:02.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:02.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:02.152 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:02.151+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:02.228 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.229 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:02.253 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:02.252+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:02.354 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:02.353+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:02.407 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.408 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:02.459 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:02.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:02.556 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:02.555+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:02.609 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.609 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:02.667 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:02.665+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:02.749 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.749 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.757 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:02.755+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:02.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:02.785 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:02.858 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:02.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:02.914 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:02.914 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:02.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:02.966 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:02.963+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:02.987 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:02.988 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:02.988 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2023-10-30T06:25:02.988 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:25:02.989 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:25:02.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2023-10-30T06:25:03.095 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:03.093+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:03.110 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:03.110 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:03.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:03.154 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:03.155 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2023-10-30T06:25:03.155 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:03.161 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:03.160+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:03.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:03.263 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:03.261+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:03.286 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:03.286 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:03.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:03.364 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:03.362+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:03.429 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:03.429 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:03.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:03.465 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:03.463+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:03.479 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:03.479 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:25:03.565 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:03.564+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:03.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.627 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:03.628 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:03.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:03.677 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:03.676+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:03.688 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:03.689 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.768 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:03.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:03.777 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:03.778 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.813 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:03.813 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:03.814 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:03.814 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:03.814 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:03.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:03.893 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:03.894 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.894 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:03.892+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:03.970 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:03.969+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:03.979 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:03.979 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:03.994 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:03.994 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:04.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:04.068 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:04.068 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:04.071 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:04.070+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:04.132 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:04.132 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:04.132 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:04.132 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:04.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:04.170 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:04.170 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:04.172 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:04.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:04.273 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:04.272+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:04.309 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:04.310 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:04.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:04.374 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:04.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:04.446 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:04.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:04.475 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:04.474+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:04.575 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:04.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:04.621 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:04.622 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:04.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:04.676 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:04.675+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:04.745 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:25:04.778 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:04.776+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:04.778 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:04.779 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:04.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:04.879 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:04.877+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:04.980 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:04.978+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:05.012 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 52.854649989000002, 2023-10-30T06:25:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 49.941476125999998, 2023-10-30T06:25:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 49.281298157000002, 2023-10-30T06:25:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 49.205874950000002, 2023-10-30T06:25:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.934751818999999, 2023-10-30T06:25:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.904472108999997, 2023-10-30T06:25:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.530561024999997, 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:05.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:05.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.526062324000002, 2023-10-30T06:25:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.234458385000003, 2023-10-30T06:25:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.228521067999999, 2023-10-30T06:25:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.227222052999998, 2023-10-30T06:25:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 48.124233244999999, 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.852329978, 2023-10-30T06:25:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.844831786, 2023-10-30T06:25:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.844170581, 2023-10-30T06:25:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.831584743000001, 2023-10-30T06:25:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.825492529999998, 2023-10-30T06:25:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 47.824497188999999, 2023-10-30T06:25:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 40.501824618000001, 2023-10-30T06:25:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 40.468383133000003, 2023-10-30T06:25:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:05.249 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:05.251 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:05.252 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:05.249+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:05.352 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:05.350+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:05.421 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:05.422 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:05.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:05.453 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:05.452+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:05.554 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:05.553+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:05.558 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:05.559 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:05.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:05.655 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:05.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:05.754 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:05.755 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:05.757 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:05.755+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:05.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:05.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:05.858 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:05.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:05.893 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:05.894 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:05.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:05.987 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:05.985+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:06.060 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:06.058+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:06.068 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:06.069 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:06.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:06.160 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:06.158+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:06.185 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:06.243 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:06.244 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:06.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:06.261 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:06.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:06.362 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:06.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:06.430 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:06.470 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:06.471 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.471 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:06.469+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:06.472 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:06.472 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:06.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:06.564 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:06.563+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:06.595 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:06.596 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.650 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:06.651 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:06.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:06.665 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:06.663+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:06.681 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:06.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:25:06.766 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:06.765+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:06.786 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:06.786 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.805 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:06.806 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:06.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:06.859 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.459 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:06.867 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:06.866+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:06.886 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:06.887 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.972 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:06.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:06.978 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:06.978 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:06.986 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:06.986 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:06.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:07.060 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:07.061 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:07.070 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:07.068+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:07.100 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.699 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:07.148 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:07.149 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:07.172 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:07.172 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:07.172 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:07.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:07.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:07.271 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:07.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:07.352 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:07.352 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:07.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:07.403 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:07.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:07.471 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:07.470+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:07.534 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:07.534 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:07.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:07.587 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:07.585+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:07.672 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:07.673 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:07.673 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:07.671+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:07.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:07.773 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:07.772+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:07.874 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:07.873+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:07.884 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:07.884 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:07.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:07.927 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:25:07.982 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:07.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:08.056 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.057 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.065 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:08.077 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:08.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:08.166 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:08.167 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:08.167 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-10-30T06:25:08.167 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2023-10-30T06:25:08.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2023-10-30T06:25:08.178 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:08.177+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:08.265 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.265 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:08.278 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:08.277+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:08.392 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:08.390+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:08.405 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.406 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:08.480 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:08.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:08.544 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.544 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.553 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:08.581 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:08.580+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:08.682 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:08.681+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:08.719 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.719 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:08.783 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:08.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:08.868 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:08.869 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:08.875 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2023-10-30T06:25:08.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:08.884 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:08.882+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:08.985 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:08.983+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:09.015 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.015 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:09.086 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:09.085+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:09.151 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.153 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:09.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:09.209 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:09.207+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:09.288 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:09.286+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:09.295 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.295 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:09.389 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:09.387+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:09.431 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:09.432 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.443 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.444 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:09.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:09.493 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:09.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:09.495 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:04.681 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.500 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:04.686 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.519 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:09.520 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.546 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:04.732 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.582 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.784 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.583 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.584 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.589 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.791 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.590 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:09.589+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:09.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:09.626 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:09.626 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.647 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.849 86024== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:09.691 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:09.690+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:09.723 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:09.723 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.781 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.781 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:09.792 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:09.790+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:09.824 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:09.825 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.892 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:09.891+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:09.910 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:09.911 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:09.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:25:09.960 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:09.961 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:09.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:10.002 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:10.002 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:10.013 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:10.012+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:10.094 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:10.093+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:10.114 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:10.150 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:10.151 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:10.151 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:10.151 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:10.151 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:10.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:10.200 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:10.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:10.297 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:10.295+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:10.328 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:10.328 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:10.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:10.328 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:10.329 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:10.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:10.354 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:10.411 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:10.409+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:10.482 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:10.482 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:10.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:10.499 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:10.497+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:10.600 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:10.598+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:10.676 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:10.677 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:10.677 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:10.677 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:10.677 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:10.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:10.700 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:10.699+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:10.800 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:10.799+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:10.814 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:10.815 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:10.815 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:10.822 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.422 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:10.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:10.880 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.480 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:10.902 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:10.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:10.992 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:10.992 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:10.992 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:10.993 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:11.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:11.003 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:11.001+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:11.103 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:11.101+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:11.126 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.726 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:11.170 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:11.171 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:11.171 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:11.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:11.224 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:11.222+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:11.305 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:11.303+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:11.398 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:11.399 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:11.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:11.399 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:11.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 58.043953842999997, 2023-10-30T06:25:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:11.400 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.401 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:11.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:11.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:11.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:11.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:11.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:11.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:11.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:11.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:11.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:11.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:11.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:11.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:11.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:11.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:11.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:11.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:11.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:11.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.722798652000002, 2023-10-30T06:25:11.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:11.420 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.420 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.420 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.420 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.420 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:11.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:11.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:11.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:11.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:11.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:11.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:11.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:11.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:11.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:11.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:11.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:11.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:11.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:11.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:11.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:11.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:11.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:11.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.434 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.718554095000002, 2023-10-30T06:25:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:11.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:11.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:11.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:11.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:11.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:11.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:11.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:11.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:11.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:11.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:11.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:11.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:11.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.449 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.633095670000003, 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.451 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:11.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.451 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:11.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:11.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:11.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:11.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:11.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:11.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:11.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:11.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:11.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:11.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:11.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:11.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:11.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:11.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:11.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:11.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.363852454000003, 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:11.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.466 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:11.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:11.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:11.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.363021043000003, 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.362353916000004, 2023-10-30T06:25:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.340079844999998, 2023-10-30T06:25:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.339431988999998, 2023-10-30T06:25:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 55.338745916000001, 2023-10-30T06:25:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.267863947999999, 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.267199982000001, 2023-10-30T06:25:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 54.238746173000003, 2023-10-30T06:25:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.896979680000001, 2023-10-30T06:25:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.896231010000001, 2023-10-30T06:25:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.895562626999997, 2023-10-30T06:25:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.885711952999998, 2023-10-30T06:25:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:11.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:11.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:11.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:11.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:11.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:11.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:11.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:11.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:11.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.885038137999999, 2023-10-30T06:25:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:11.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:11.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:11.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:11.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:11.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:11.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:11.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:11.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.677 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:11.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.884389755000001, 2023-10-30T06:25:11.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:11.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:11.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:11.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:11.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:11.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:11.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:11.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:11.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:11.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:11.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:11.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:11.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:11.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:11.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:11.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:11.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:11.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:11.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.880273289000002, 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:11.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:11.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:11.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:11.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:11.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:11.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:11.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:11.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:11.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:11.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:11.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:11.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:11.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:11.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:11.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:11.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:11.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:11.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:11.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:11.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:11.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:11.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:11.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:11.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:11.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:11.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:11.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:11.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:11.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:11.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:11.708 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:11.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:11.710 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:25:11.711 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:11.708+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:11.811 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:11.810+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:11.881 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:11.882 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:11.882 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:11.882 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:11.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:11.912 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:11.910+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:12.013 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:12.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:12.017 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:12.017 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:12.018 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:12.018 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:12.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:12.114 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:12.113+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:12.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:12.196 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:12.196 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:12.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:12.197 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:12.197 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:12.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:12.250 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:12.248+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:12.316 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:12.315+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:12.350 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:12.350 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:12.350 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:12.351 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:12.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:12.374 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:12.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.417 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:12.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:12.460 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.518 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:12.517+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:12.531 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:12.532 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:12.532 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:12.540 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:12.584 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:12.584 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.620 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:12.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:12.708 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:12.708 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.709 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:12.709 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:12.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:12.720 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:12.718+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:12.795 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:12.796 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.821 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:12.820+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:12.854 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:12.855 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:12.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:12.890 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:12.890 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.921 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:12.920+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:12.977 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:12.978 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:12.993 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:12.993 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:13.043 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:13.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:13.063 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:13.064 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:13.123 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:13.122+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:13.169 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.169 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:13.224 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:13.222+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:13.306 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.306 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:13.358 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:13.357+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:13.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:13.426 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:13.424+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:13.483 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.484 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:13.539 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:13.537+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:13.626 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.627 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:13.640 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:13.638+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:13.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:25:13.757 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:13.755+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:13.786 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.787 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:13.830 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:13.829+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:13.888 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:13.889 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:13.889 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-10-30T06:25:13.890 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2023-10-30T06:25:13.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2023-10-30T06:25:13.931 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:13.931 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:13.935 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:13.933+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:13.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:14.032 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:14.030+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:14.068 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.069 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:14.133 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:14.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:14.146 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:14.209 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.209 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:14.234 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:14.232+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:14.335 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:14.334+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:14.347 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.347 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:14.437 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:14.436+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:14.496 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.496 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:14.547 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:14.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:14.634 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.634 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.639 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:14.638+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:14.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:14.740 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:14.738+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:14.768 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.768 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:25:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 0 (802) 2023-10-30T06:25:14.880 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:14.878+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:14.942 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:14.940+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:14.956 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:14.956 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:14.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:15.043 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:15.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:15.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:15.136 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:15.136 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:15.144 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:15.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:15.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:15.262 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:15.259+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:15.278 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:15.279 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:15.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:15.326 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:15.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.433 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:15.434 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.434 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:15.432+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:15.447 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:15.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:15.518 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.531 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:15.532 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:15.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:15.532 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:15.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.532 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:15.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:15.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 62.191045228999997, 2023-10-30T06:25:15.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:15.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:15.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:15.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:15.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:15.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:15.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:15.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:15.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:15.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:15.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:15.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:15.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:15.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:15.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:15.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:15.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:15.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:15.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:15.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.553 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:15.553 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.869890038000001, 2023-10-30T06:25:15.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:15.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:15.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:15.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:15.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:15.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:15.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:15.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:15.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:15.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:15.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:15.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:15.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:15.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:15.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:15.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:15.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:15.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:15.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:15.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:15.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.570 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.570 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:15.570 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.865645481000001, 2023-10-30T06:25:15.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:15.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:15.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:15.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:15.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:15.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:15.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:15.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:15.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:15.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:15.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:15.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:15.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:15.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:15.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:15.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:15.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:15.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:15.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:15.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:15.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:15.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:15.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:15.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:15.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:15.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.780187056000003, 2023-10-30T06:25:15.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:15.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:15.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.591 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:15.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:15.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:15.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:15.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:15.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:15.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:15.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:15.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:15.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:15.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:15.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:15.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:15.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:15.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:15.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:15.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:15.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:15.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:15.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:15.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:15.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:15.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:15.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:15.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:15.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.510943840000003, 2023-10-30T06:25:15.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:15.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:15.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.614 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:15.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:15.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:15.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:15.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:15.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:15.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:15.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:15.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:15.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:15.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:15.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:15.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:15.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:15.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:15.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:15.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:15.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:15.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:15.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:15.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:15.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:15.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:15.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:15.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.510112429000003, 2023-10-30T06:25:15.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:15.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:15.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:15.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:15.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:15.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:15.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:15.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:15.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:15.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:15.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:15.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:15.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:15.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:15.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:15.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:15.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:15.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:15.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:15.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:15.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.509445302000003, 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:15.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.654 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:15.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:15.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:15.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:15.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:15.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:15.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:15.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:15.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:15.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:15.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:15.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:15.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:15.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:15.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:15.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:15.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:15.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:15.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:15.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:15.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:15.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:15.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.487171230999998, 2023-10-30T06:25:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:15.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:15.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.675 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:15.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:15.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:15.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:15.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:15.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:15.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:15.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:15.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:15.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:15.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:15.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:15.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:15.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:15.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:15.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:15.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:15.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:15.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:15.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:15.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:15.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:15.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.486523374999997, 2023-10-30T06:25:15.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:15.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:15.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.696 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:15.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:15.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:15.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:15.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:15.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:15.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:15.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:15.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:15.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:15.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:15.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:15.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:15.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:15.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:15.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.710 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.711 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:15.711 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:15.711 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.485837302, 2023-10-30T06:25:15.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:15.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:15.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.715 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:15.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:15.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:15.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:15.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:15.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:15.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:15.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:15.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:15.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:15.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:15.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:15.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:15.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:15.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:15.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:15.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:15.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:15.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:15.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:15.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:15.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:15.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.733 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.733 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:15.734 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:15.734 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 58.414955333999998, 2023-10-30T06:25:15.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:15.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.734 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.735 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.735 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.735 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.735 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:15.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.736 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:15.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:15.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:15.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:15.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:15.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:15.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:15.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:15.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:15.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:15.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:15.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:15.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:15.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:15.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:15.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:15.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:15.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:15.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:15.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:15.748 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.749 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:15.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:15.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 58.414291368000001, 2023-10-30T06:25:15.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:15.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.750 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.751 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:15.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:15.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:15.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:15.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:15.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:15.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:15.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:15.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:15.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:15.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:15.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 58.385837559000002, 2023-10-30T06:25:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:15.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.044071066000001, 2023-10-30T06:25:15.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:15.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:15.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:15.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:15.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:15.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:15.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:15.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:15.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:15.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:15.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:15.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:15.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:15.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:15.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:15.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:15.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:15.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:15.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:15.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:15.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:15.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:15.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:15.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.043322396000001, 2023-10-30T06:25:15.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:15.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:15.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:15.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:15.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:15.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:15.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:15.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:15.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:15.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:15.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:15.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:15.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:15.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:15.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:15.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:15.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:15.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:15.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:15.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:15.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:15.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:15.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.822 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.822 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:15.822 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.042654013000003, 2023-10-30T06:25:15.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:15.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.823 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.823 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:15.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:15.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:15.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:15.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:15.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:15.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:15.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:15.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:15.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:15.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:15.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:15.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:15.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:15.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:15.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:15.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:15.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:15.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:15.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:15.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:15.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:15.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:15.842 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.032803338999997, 2023-10-30T06:25:15.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:15.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:15.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:15.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:15.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:15.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:15.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:15.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:15.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:15.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:15.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:15.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:15.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:15.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:15.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:15.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:15.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:15.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:15.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:15.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:15.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:15.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:15.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:15.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:15.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:15.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:15.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:15.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.032129523999998, 2023-10-30T06:25:15.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:15.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:15.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:15.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:15.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:15.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:15.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:15.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:15.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:15.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:15.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:15.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:15.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:15.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:15.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:15.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:15.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:15.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:15.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:15.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:15.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:15.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:15.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:15.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:15.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:15.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:15.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:15.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.031481141, 2023-10-30T06:25:15.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:15.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.897 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:15.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:15.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:15.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:15.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:15.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:15.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:15.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:15.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:15.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:15.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:15.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:15.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:15.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:15.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:15.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:15.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:15.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:15.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:15.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:15.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:15.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:15.913 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:15.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 51.027364675000001, 2023-10-30T06:25:15.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:15.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:15.914 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:15.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:15.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:15.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:15.915 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:15.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.915 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:15.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:15.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:15.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:15.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:15.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:15.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:15.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:15.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:15.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:15.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:15.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:15.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:15.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:15.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:15.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:15.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:15.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:15.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:15.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:15.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:15.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:15.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:15.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:15.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:15.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:15.929 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:15.930 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:15.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:15.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:15.933 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.934 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:15.934 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.934 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.935 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:15.936 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:15.930+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:15.958 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:15.959 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:16.032 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:16.030+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:16.093 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:16.093 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:16.094 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:16.132 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:16.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:16.233 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:16.232+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:16.293 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.293 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:16.294 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:16.294 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:16.294 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:16.334 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:16.333+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:16.436 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:16.434+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:16.469 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.470 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:16.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:16.470 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:16.470 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:16.536 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:16.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:16.638 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:16.636+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:16.651 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.652 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:16.652 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:16.652 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:16.738 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:16.736+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:16.787 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.787 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:16.787 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:16.788 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:16.788 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:16.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:25:16.838 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:16.836+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:16.939 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:16.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:16.977 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:16.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:16.978 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:16.986 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:17.039 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:17.037+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:17.129 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.129 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:17.130 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:17.130 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:17.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:17.140 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:17.138+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:17.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:17.241 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:17.239+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:17.307 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:17.308 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:17.308 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:17.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:17.341 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:17.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:17.442 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:17.441+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:17.445 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.445 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:17.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:17.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:17.446 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:17.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:17.544 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:17.542+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:17.581 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.581 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:17.582 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:17.582 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:17.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:17.675 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:17.674+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:17.746 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:17.745+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:17.759 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.759 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:17.759 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:17.759 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:17.759 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:17.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:17.850 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:17.852 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:17.851+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:17.950 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:17.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:17.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:17.975 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:17.975 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:17.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:17.976 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:17.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.976 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:17.976 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:17.976 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.820004804999996, 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:17.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:17.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:17.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.978 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:17.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:17.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:17.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:17.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:17.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:17.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:17.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:17.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:17.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:17.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:17.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:17.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:17.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:17.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:17.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:17.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:17.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:17.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:17.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:17.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:17.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:17.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:17.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:17.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:17.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:17.989 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:17.989 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:17.989 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:17.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.990 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:17.990 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 62.906830941999999, 2023-10-30T06:25:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:17.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:17.992 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:17.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:17.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:17.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:17.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:17.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:17.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:17.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:17.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:17.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:17.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:17.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:17.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:17.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:17.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:17.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:17.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:17.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:17.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:17.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:17.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:17.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:18.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:18.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:18.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:18.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:18.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:18.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:18.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:18.004 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.005 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.005 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.006 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:18.006 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:18.006 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 62.246652973000003, 2023-10-30T06:25:18.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:18.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.008 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:18.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:18.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:18.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:18.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:18.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:18.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:18.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:18.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:18.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:18.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:18.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:18.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:18.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:18.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:18.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:18.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.018 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.019 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:18.019 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:18.020 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 62.171229766000003, 2023-10-30T06:25:18.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:18.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.020 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:18.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:18.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:18.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:18.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:18.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:18.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:18.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:18.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:18.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:18.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:18.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:18.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:18.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:18.032 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.900106635, 2023-10-30T06:25:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.033 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:18.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:18.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:18.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:18.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:18.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:18.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.044 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:18.045 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.869826924999998, 2023-10-30T06:25:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:18.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:18.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:18.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:18.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:18.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.057 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:18.057 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:18.057 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.495915840999999, 2023-10-30T06:25:18.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:18.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.058 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:18.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:18.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:18.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:18.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:18.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:18.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:18.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:18.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:18.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:18.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:18.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:18.067 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.068 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.068 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.068 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.491417140000003, 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.069 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:18.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:18.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:18.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:18.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:18.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:18.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:18.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:18.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:18.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:18.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:18.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:18.079 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.079 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.079 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.199813200999998, 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:18.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.081 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:18.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:18.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:18.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:18.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:18.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:18.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:18.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:18.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:18.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:18.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:18.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:18.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:18.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:18.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:18.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.091 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:18.092 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:18.092 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.193875884000001, 2023-10-30T06:25:18.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:18.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:18.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:18.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:18.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:18.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:18.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:18.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:18.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:18.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:18.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.192576869, 2023-10-30T06:25:18.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:18.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.104 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:18.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:18.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:18.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:18.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:18.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:18.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:18.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:18.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:18.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:18.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:18.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:18.113 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.114 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.114 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.114 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:18.115 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:18.115 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 61.089588061000001, 2023-10-30T06:25:18.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:18.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:18.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:18.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:18.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:18.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:18.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:18.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:18.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:18.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:18.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.126 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:18.126 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:18.126 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.817684794000002, 2023-10-30T06:25:18.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:18.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.127 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:18.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:18.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:18.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:18.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:18.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:18.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:18.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:18.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:18.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:18.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:18.136 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.810186602000002, 2023-10-30T06:25:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.809525397000002, 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:18.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:18.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:18.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:18.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:18.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:18.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:18.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.161 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.796939559000002, 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.163 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:18.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:18.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:18.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:18.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:18.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:18.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:18.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:18.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:18.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:18.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:18.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.172 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.790847346, 2023-10-30T06:25:18.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:18.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.174 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:18.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:18.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:18.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:18.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:18.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:18.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:18.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:18.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:18.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:18.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:18.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:18.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.184 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 60.789852005, 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.185 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:18.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 53.467179434000002, 2023-10-30T06:25:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:18.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:18.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:18.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:18.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:18.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:18.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:18.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:18.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:18.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:18.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.207 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:18.207 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:18.207 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 53.433737948999998, 2023-10-30T06:25:18.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:18.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:18.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:18.208 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:18.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:18.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:18.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:18.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:18.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:18.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:18.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:18.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:18.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:18.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:18.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:18.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:18.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:18.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:18.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:18.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:18.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:18.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:18.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:18.218 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:18.218 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:18.219 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.220 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:18.219+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:18.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:18.286 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:18.286 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.321 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:18.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:18.323 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:25:18.377 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:18.377 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.394 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:18.395 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:18.395 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:18.395 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:18.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:18.462 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:18.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.463 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:18.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:18.522 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:18.522+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:18.531 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:18.531 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:18.531 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:18.532 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:18.532 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:18.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:18.548 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:18.548 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.624 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:18.623+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:18.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:18.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.679 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:18.679 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:18.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:18.680 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:18.680 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:18.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:18.737 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:18.736+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:18.750 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.826 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:18.825+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:18.858 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:18.859 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:18.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:18.926 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:18.925+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:18.996 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:18.997 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.006 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:19.051 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:19.049+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:19.127 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:19.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.132 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:19.229 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:19.228+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:19.271 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.271 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:19.330 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:19.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:19.424 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.425 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.431 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:19.430+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:19.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:19.531 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:19.531+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:19.605 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.605 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:19.655 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:19.655+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:19.733 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:19.732+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:19.780 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.780 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:19.833 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:19.833+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:19.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:19.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:19.839 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:25:19.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:25:19.934 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:19.934+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:19.959 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:19.959 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:19.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:20.035 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:20.034+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:20.137 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:20.136+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:20.138 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:20.139 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:20.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:20.238 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:20.236+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:20.315 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:20.316 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:20.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:20.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:25:20.369 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:20.368+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:20.439 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:20.438+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:20.454 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:20.455 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:20.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:20.478 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:20.479 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:20.479 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 35, 2023-10-30T06:25:20.479 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:25:20.479 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:25:20.479 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:25:18.240347+0000", 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:25:20.480 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:25:20.481 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:25:20.481 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:25:20.481 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:25:20.481 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:20.481 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:25:20.482 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:25:20.482 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:25:20.482 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:25:20.482 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:25:20.482 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:25:20.483 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:25:20.483 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:25:20.483 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:25:20.483 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:25:20.483 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:25:20.484 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:25:20.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.484 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:25:20.484 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:25:20.484 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:25:20.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:25:20.485 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:25:20.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:25:20.485 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:25:20.485 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:25:20.486 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:25:20.486 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:25:20.486 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:25:20.486 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:25:20.486 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:25:20.487 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:25:20.487 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:25:20.487 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:25:20.487 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:25:20.487 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:25:20.488 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:25:20.488 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:25:20.488 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:25:20.488 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:25:20.488 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:25:20.489 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:25:20.489 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:25:20.489 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:25:20.489 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:25:20.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.490 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:25:20.490 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:25:20.490 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:25:20.490 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:25:20.491 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:25:20.491 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:25:20.491 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:25:20.491 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:25:20.491 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:25:20.492 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:25:20.493 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:25:20.493 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:25:20.493 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:25:20.493 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:25:20.493 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:25:20.494 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:25:20.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:25:20.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.495 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:25:20.495 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:25:20.495 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:25:20.496 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:25:20.496 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:25:20.496 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:25:20.496 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:25:20.496 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:25:20.497 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:25:20.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.498 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:25:20.498 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:25:20.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.498 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:25:20.499 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:25:20.499 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:25:20.499 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:25:20.499 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:25:20.499 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.501 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:25:20.501 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:25:20.501 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:25:20.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:25:20.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:25:20.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:25:20.502 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:25:20.502 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:25:20.502 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:25:20.503 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:25:20.503 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:25:20.503 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:25:20.503 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:25:20.504 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:25:20.504 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:25:20.504 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:25:20.504 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:25:20.505 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:25:20.505 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:25:20.505 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:25:20.505 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:25:20.505 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:25:20.506 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:25:20.506 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:25:20.506 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:25:20.506 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:25:20.506 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:25:20.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.507 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:25:20.507 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:25:20.507 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:25:20.507 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:25:20.508 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:25:20.509 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:25:20.509 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:25:20.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:25:20.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:25:20.509 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:25:20.510 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:25:20.510 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:25:20.510 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:25:20.510 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:25:20.511 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:25:20.511 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:25:20.511 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:25:20.511 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:25:20.511 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:25:20.512 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:25:20.512 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:25:20.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:25:20.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.512 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:25:20.513 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:25:20.513 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:25:20.513 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:25:20.513 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:25:20.513 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:25:20.514 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:25:20.515 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:25:20.515 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:25:20.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.515 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:25:20.516 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 3.5099999904632568, 2023-10-30T06:25:20.516 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 3.5099999904632568, 2023-10-30T06:25:20.516 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:25:20.516 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 4, 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 4, 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.518 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:20.518 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:25:20.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.518 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:25:20.518 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:25:20.519 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.519 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.519 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.519 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.519 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.520 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.521 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.521 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:25:20.521 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:25:20.522 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.524 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.524 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:25:20.524 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:25:20.527 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:25:20.528 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.529 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:25:20.530 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.530 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:25:20.531 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:20.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.531 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:25:20.532 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:25:20.532 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:25:20.532 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:25:20.532 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.532 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.533 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.533 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:25:20.534 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:25:20.534 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.534 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.534 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.534 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.535 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.535 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.535 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:25:20.535 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 33, 2023-10-30T06:25:20.535 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.536 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.536 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.536 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.537 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:25:20.537 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.538 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:25:20.538 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.538 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.538 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.539 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:25:20.540 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.541 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.541 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.542 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:25:20.542 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.543 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.544 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.544 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:25:20.545 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:25:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.547 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.548 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.548 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:25:20.549 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.550 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.550 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:20.550 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 26, 2023-10-30T06:25:20.550 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.550 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.551 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.551 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.551 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:25:20.552 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.552 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.553 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.554 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:25:20.554 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.555 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.555 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:25:20.555 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.557 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:25:20.557 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.558 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:25:20.559 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.560 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:25:20.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:25:20.561 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:25:20.562 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:25:20.562 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.562 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.562 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.563 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.564 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.564 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.564 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.564 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.564 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:20.565 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:20.565 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.565 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.565 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.565 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.566 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:25:20.566 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.568 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.568 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.569 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:25:20.570 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.570 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:25:20.571 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.572 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:25:20.572 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.573 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.573 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.574 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:25:20.574 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.576 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:25:20.576 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:25:20.576 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:25:20.576 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.578 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:25:20.578 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:25:20.578 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.578 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.578 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:20.579 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.580 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.580 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.580 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.581 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:25:20.581 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.582 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:25:20.582 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:25:20.583 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.584 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:25:20.584 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.585 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.585 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:25:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:25:20.590 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:20.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.590 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.590 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:25:20.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:25:20.591 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:25:20.591 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:25:20.591 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.591 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:25:20.592 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.593 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.594 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:20.594 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:20.594 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.594 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.594 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.595 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.595 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:25:20.595 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:25:20.596 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.598 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:25:20.598 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.599 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:25:20.599 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.600 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:25:20.601 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.601 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:25:20.602 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.602 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.602 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:25:20.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:20.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.607 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.608 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.608 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.608 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.609 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.610 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.610 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:20.610 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:25:20.610 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.610 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.611 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.611 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:25:20.612 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.613 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.613 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.614 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.614 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:25:20.614 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.615 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:25:20.615 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.617 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:25:20.617 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:25:20.618 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.618 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:25:20.619 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:25:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:20.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.621 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:25:20.621 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:25:20.622 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:25:20.622 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:25:20.622 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.622 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.622 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:25:20.623 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:20.624 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:20.625 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:20.625 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:20.625 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:20.625 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:20.625 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.626 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:25:20.626 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.627 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:25:20.627 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.629 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:25:20.629 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.630 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:20.631 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.631 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:25:20.631 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.632 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:25:20.632 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.633 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:20.633 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:20.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.634 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:20.634 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:25:20.634 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.636 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:25:20.636 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:25:20.636 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:25:20.636 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.637 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:20.638 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:25:20.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.638 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:25:20.638 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.638 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.639 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.639 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.639 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.639 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:25:20.639 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.640 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:25:20.640 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.640 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.642 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:25:20.642 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.642 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.642 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.644 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:25:20.644 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.644 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.644 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.644 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:25:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.646 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:25:20.646 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.646 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.646 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.646 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.647 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:25:20.648 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.648 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.648 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.648 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.649 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:25:20.649 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.650 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:25:20.650 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.650 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.650 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.651 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.651 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.651 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:25:20.651 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.652 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:25:20.652 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:20.652 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:20.652 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:20.653 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:20.653 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:20.653 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:25:20.653 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:20.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:25:20.654 INFO:teuthology.orchestra.run.smithi079.stdout: 2, 2023-10-30T06:25:20.655 INFO:teuthology.orchestra.run.smithi079.stdout: 1 2023-10-30T06:25:20.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:20.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.655 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:20.655 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [], 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:25:20.656 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:25:20.657 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:25:20.657 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:25:20.657 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:25:20.657 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:25:20.657 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:25:20.658 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:25:20.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:20.658 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:25:20.658 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:25:20.659 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:25:20.660 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:25:20.660 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:25:20.660 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:25:20.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:20.660 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:20.663 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:25:18.240347+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.509999990463257, 'score_stable': 3.509999990463257, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 4, 'raw_score_stable': 4, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [2, 1]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:25:20.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:25:20.664 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:20.664 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:20.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:20.665 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:20.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:20.763 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:20.762+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:20.851 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:20.851 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:20.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:20.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:20.864 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:20.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:20.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:20.964 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:20.963+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:21.036 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.036 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:21.065 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:21.064+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:21.167 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:21.166+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:21.219 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.220 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:21.270 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:21.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:21.299 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:21.300 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:25:21.311 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-10-30T06:25:21.268696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219060,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755660,"statfs":{"total":773094113280,"available":772869795840,"internally_reserved":0,"allocated":2965504,"data_stored":1883466,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.396917"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-30T06:25:15.863135+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:25:15.863135+0000","last_peered":"2023-10-30T06:25:15.863135+0000","last_clean":"2023-10-30T06:25:15.863135+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:25:15.863135+0000","last_undegraded":"2023-10-30T06:25:15.863135+0000","last_fullsized":"2023-10-30T06:25:15.863135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-30T06:25:16.524688+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:25:16.524688+0000","last_peered":"2023-10-30T06:25:16.524688+0000","last_clean":"2023-10-30T06:25:16.524688+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:25:16.524688+0000","last_undegraded":"2023-10-30T06:25:16.524688+0000","last_fullsized":"2023-10-30T06:25:16.524688+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-30T06:25:16.272269+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:25:16.272269+0000","last_peered":"2023-10-30T06:25:16.272269+0000","last_clean":"2023-10-30T06:25:16.272269+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:25:16.272269+0000","last_undegraded":"2023-10-30T06:25:16.272269+0000","last_fullsized":"2023-10-30T06:25:16.272269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-30T06:25:16.008994+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:25:16.008994+0000","last_peered":"2023-10-30T06:25:16.008994+0000","last_clean":"2023-10-30T06:25:16.008994+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:25:16.008994+0000","last_undegraded":"2023-10-30T06:25:16.008994+0000","last_fullsized":"2023-10-30T06:25:16.008994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-30T06:25:15.861317+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:25:15.861317+0000","last_peered":"2023-10-30T06:25:15.861317+0000","last_clean":"2023-10-30T06:25:15.861317+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:25:15.861317+0000","last_undegraded":"2023-10-30T06:25:15.861317+0000","last_fullsized":"2023-10-30T06:25:15.861317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-30T06:25:16.037022+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:25:16.037022+0000","last_peered":"2023-10-30T06:25:16.037022+0000","last_clean":"2023-10-30T06:25:16.037022+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:25:16.037022+0000","last_undegraded":"2023-10-30T06:25:16.037022+0000","last_fullsized":"2023-10-30T06:25:16.037022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-30T06:25:15.856292+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:25:15.856292+0000","last_peered":"2023-10-30T06:25:15.856292+0000","last_clean":"2023-10-30T06:25:15.856292+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:25:15.856292+0000","last_undegraded":"2023-10-30T06:25:15.856292+0000","last_fullsized":"2023-10-30T06:25:15.856292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-30T06:25:15.859750+0000","last_change":"2023-10-30T06:24:20.400180+0000","last_active":"2023-10-30T06:25:15.859750+0000","last_peered":"2023-10-30T06:25:15.859750+0000","last_clean":"2023-10-30T06:25:15.859750+0000","last_became_active":"2023-10-30T06:24:20.390956+0000","last_became_peered":"2023-10-30T06:24:20.390956+0000","last_unstale":"2023-10-30T06:25:15.859750+0000","last_undegraded":"2023-10-30T06:25:15.859750+0000","last_fullsized":"2023-10-30T06:25:15.859750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:50:38.971763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-30T06:25:15.049966+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:25:15.049966+0000","last_peered":"2023-10-30T06:25:15.049966+0000","last_clean":"2023-10-30T06:25:15.049966+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:25:15.049966+0000","last_undegraded":"2023-10-30T06:25:15.049966+0000","last_fullsized":"2023-10-30T06:25:15.049966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411345,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":535186,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.359999999999999,"5min":10.359999999999999,"15min":10.359999999999999},"min":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":24.350000000000001,"5min":24.350000000000001,"15min":24.350000000000001},"last":6.444},{"interface":"front","average":{"1min":12.083,"5min":12.083,"15min":12.083},"min":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"max":{"1min":46.798000000000002,"5min":46.798000000000002,"15min":46.798000000000002},"last":12.441000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.798,"5min":13.798,"15min":13.798},"min":{"1min":5.5209999999999999,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":52.552,"5min":52.552,"15min":52.552},"last":14.298},{"interface":"front","average":{"1min":15.147,"5min":15.147,"15min":15.147},"min":{"1min":7.1589999999999998,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":27.946000000000002,"5min":27.946000000000002,"15min":27.946000000000002},"last":12.109999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"min":{"1min":11.013,"5min":11.013,"15min":11.013},"max":{"1min":118.45699999999999,"5min":118.45699999999999,"15min":118.45699999999999},"last":10.119999999999999},{"interface":"front","average":{"1min":22.571999999999999,"5min":22.571999999999999,"15min":22.571999999999999},"min":{"1min":7.8739999999999997,"5min":7.8739999999999997,"15min":7.8739999999999997},"max":{"1min":124.075,"5min":124.075,"15min":124.075},"last":15.819000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.675000000000001,"5min":13.675000000000001,"15min":13.675000000000001},"min":{"1min":7.2359999999999998,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":51.649000000000001,"5min":51.649000000000001,"15min":51.649000000000001},"last":10.679},{"interface":"front","average":{"1min":17.998999999999999,"5min":17.998999999999999,"15min":17.998999999999999},"min":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":38.271999999999998,"5min":38.271999999999998,"15min":38.271999999999998},"last":15.023999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.052,"5min":21.052,"15min":21.052},"min":{"1min":10.776,"5min":10.776,"15min":10.776},"max":{"1min":123.754,"5min":123.754,"15min":123.754},"last":17.625},{"interface":"front","average":{"1min":23.079999999999998,"5min":23.079999999999998,"15min":23.079999999999998},"min":{"1min":8.5679999999999996,"5min":8.5679999999999996,"15min":8.5679999999999996},"max":{"1min":122.72499999999999,"5min":122.72499999999999,"15min":122.72499999999999},"last":7.6619999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.457000000000001,"5min":21.457000000000001,"15min":21.457000000000001},"min":{"1min":11.9,"5min":11.9,"15min":11.9},"max":{"1min":66.400999999999996,"5min":66.400999999999996,"15min":66.400999999999996},"last":21.195},{"interface":"front","average":{"1min":19.859000000000002,"5min":19.859000000000002,"15min":19.859000000000002},"min":{"1min":8.2070000000000007,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":53.149000000000001,"5min":53.149000000000001,"15min":53.149000000000001},"last":18.427}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.513999999999999,"5min":21.513999999999999,"15min":21.513999999999999},"min":{"1min":10.657,"5min":10.657,"15min":10.657},"max":{"1min":45.235999999999997,"5min":45.235999999999997,"15min":45.235999999999997},"last":13.106999999999999},{"interface":"front","average":{"1min":21.594999999999999,"5min":21.594999999999999,"15min":21.594999999999999},"min":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":69.991,"5min":69.991,"15min":69.991},"last":20.555}]}]},{"osd":6,"up_from":16,"seq":68719476754,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":131867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":14.706,"5min":14.706,"15min":14.706},"min":{"1min":4.8300000000000001,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":94.343000000000004,"5min":94.343000000000004,"15min":94.343000000000004},"last":13.340999999999999},{"interface":"front","average":{"1min":15.754,"5min":15.754,"15min":15.754},"min":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":122.054,"5min":122.054,"15min":122.054},"last":8.7129999999999992}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"min":{"1min":6.4379999999999997,"5min":6.4379999999999997,"15min":6.4379999999999997},"max":{"1min":18.992999999999999,"5min":18.992999999999999,"15min":18.992999999999999},"last":14.317},{"interface":"front","average":{"1min":33.863999999999997,"5min":33.863999999999997,"15min":33.863999999999997},"min":{"1min":5.9269999999999996,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":387.15600000000001,"5min":387.15600000000001,"15min":387.15600000000001},"last":16.978999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.956,"5min":19.956,"15min":19.956},"min":{"1min":6.4359999999999999,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":156.90299999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":19.962},{"interface":"front","average":{"1min":20.584,"5min":20.584,"15min":20.584},"min":{"1min":8.8940000000000001,"5min":8.8940000000000001,"15min":8.8940000000000001},"max":{"1min":143.578,"5min":143.578,"15min":143.578},"last":19.172000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.963000000000001,"5min":21.963000000000001,"15min":21.963000000000001},"min":{"1min":10.696999999999999,"5min":10.696999999999999,"15min":10.696999999999999},"max":{"1min":141.18799999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":21.324999999999999},{"interface":"front","average":{"1min":25.420999999999999,"5min":25.420999999999999,"15min":25.420999999999999},"min":{"1min":13.539999999999999,"5min":13.539999999999999,"15min":13.539999999999999},"max":{"1min":161.32900000000001,"5min":161.32900000000001,"15min":161.32900000000001},"last":22.91}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":15.045,"5min":15.045,"15min":15.045},"min":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"max":{"1min":74.079999999999998,"5min":74.079999999999998,"15min":74.079999999999998},"last":10.167999999999999},{"interface":"front","average":{"1min":15.702,"5min":15.702,"15min":15.702},"min":{"1min":9.2270000000000003,"5min":9.2270000000000003,"15min":9.2270000000000003},"max":{"1min":58.613999999999997,"5min":58.613999999999997,"15min":58.613999999999997},"last":11.715999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":17.966000000000001,"5min":17.966000000000001,"15min":17.966000000000001},"min":{"1min":9.6560000000000006,"5min":9.6560000000000006,"15min":9.6560000000000006},"max":{"1min":71.757000000000005,"5min":71.757000000000005,"15min":71.757000000000005},"last":14.733000000000001},{"interface":"front","average":{"1min":21.762,"5min":21.762,"15min":21.762},"min":{"1min":14.734,"5min":14.734,"15min":14.734},"max":{"1min":80.119,"5min":80.119,"15min":80.119},"last":24.132000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.741,"5min":19.741,"15min":19.741},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":81.247,"5min":81.247,"15min":81.247},"last":17.289000000000001},{"interface":"front","average":{"1min":19.088000000000001,"5min":19.088000000000001,"15min":19.088000000000001},"min":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":70.989000000000004,"5min":70.989000000000004,"15min":70.989000000000004},"last":22.119}]}]},{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":126388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":11.06,"5min":11.06,"15min":11.06},"min":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":24.466000000000001,"5min":24.466000000000001,"15min":24.466000000000001},"last":7.5659999999999998},{"interface":"front","average":{"1min":10.257999999999999,"5min":10.257999999999999,"15min":10.257999999999999},"min":{"1min":6,"5min":6,"15min":6},"max":{"1min":19.263999999999999,"5min":19.263999999999999,"15min":19.263999999999999},"last":6.9569999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":26.827000000000002,"5min":26.827000000000002,"15min":26.827000000000002},"min":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":165.76400000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":9.7870000000000008},{"interface":"front","average":{"1min":28.079000000000001,"5min":28.079000000000001,"15min":28.079000000000001},"min":{"1min":8.5760000000000005,"5min":8.5760000000000005,"15min":8.5760000000000005},"max":{"1min":190.34200000000001,"5min":190.34200000000001,"15min":190.34200000000001},"last":21.556000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":18.428000000000001,"5min":18.428000000000001,"15min":18.428000000000001},"min":{"1min":12.856999999999999,"5min":12.856999999999999,"15min":12.856999999999999},"max":{"1min":41.229999999999997,"5min":41.229999999999997,"15min":41.229999999999997},"last":23.326000000000001},{"interface":"front","average":{"1min":17.199000000000002,"5min":17.199000000000002,"15min":17.199000000000002},"min":{"1min":8.1739999999999995,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":33.060000000000002,"5min":33.060000000000002,"15min":33.060000000000002},"last":10.81}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.620000000000001,"5min":20.620000000000001,"15min":20.620000000000001},"min":{"1min":9.9420000000000002,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":42.274999999999999,"5min":42.274999999999999,"15min":42.274999999999999},"last":19.093},{"interface":"front","average":{"1min":11.614000000000001,"5min":11.614000000000001,"15min":11.614000000000001},"min":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":19.716000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.658999999999999,"5min":19.658999999999999,"15min":19.658999999999999},"min":{"1min":7.4450000000000003,"5min":7.4450000000000003,"15min":7.4450000000000003},"max":{"1min":56.796999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":22.614000000000001},{"interface":"front","average":{"1min":20.798999999999999,"5min":20.798999999999999,"15min":20.798999999999999},"min":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"max":{"1min":58.576000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":8.6620000000000008}]},{"osd":6,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":29.138000000000002,"5min":29.138000000000002,"15min":29.138000000000002},"min":{"1min":11.694000000000001,"5min":11.694000000000001,"15min":11.694000000000001},"max":{"1min":169.05199999999999,"5min":169.05199999999999,"15min":169.05199999999999},"last":41.771000000000001},{"interface":"front","average":{"1min":15.494,"5min":15.494,"15min":15.494},"min":{"1min":9.6419999999999995,"5min":9.6419999999999995,"15min":9.6419999999999995},"max":{"1min":30.957000000000001,"5min":30.957000000000001,"15min":30.957000000000001},"last":40.128999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":17.986000000000001,"5min":17.986000000000001,"15min":17.986000000000001},"min":{"1min":8.5950000000000006,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":38.887999999999998,"5min":38.887999999999998,"15min":38.887999999999998},"last":16.527999999999999},{"interface":"front","average":{"1min":29.614999999999998,"5min":29.614999999999998,"15min":29.614999999999998},"min":{"1min":10.025,"5min":10.025,"15min":10.025},"max":{"1min":170.232,"5min":170.232,"15min":170.232},"last":23.974}]}]},{"osd":2,"up_from":16,"seq":68719476752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":131867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.809000000000001,"5min":27.809000000000001,"15min":27.809000000000001},"min":{"1min":5.0730000000000004,"5min":5.0730000000000004,"15min":5.0730000000000004},"max":{"1min":258.66199999999998,"5min":258.66199999999998,"15min":258.66199999999998},"last":14.763999999999999},{"interface":"front","average":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":203.38499999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":8.9060000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":16.311,"5min":16.311,"15min":16.311},"min":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":45.936999999999998,"5min":45.936999999999998,"15min":45.936999999999998},"last":20.626999999999999},{"interface":"front","average":{"1min":14.839,"5min":14.839,"15min":14.839},"min":{"1min":6.8360000000000003,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":37.390999999999998,"5min":37.390999999999998,"15min":37.390999999999998},"last":11.663}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.326000000000001,"5min":19.326000000000001,"15min":19.326000000000001},"min":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":183.38900000000001,"5min":183.38900000000001,"15min":183.38900000000001},"last":8.1839999999999993},{"interface":"front","average":{"1min":23.067,"5min":23.067,"15min":23.067},"min":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":189.374,"5min":189.374,"15min":189.374},"last":15.358000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.959,"5min":21.959,"15min":21.959},"min":{"1min":7.7229999999999999,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":186.01400000000001,"5min":186.01400000000001,"15min":186.01400000000001},"last":12.286},{"interface":"front","average":{"1min":20.114999999999998,"5min":20.114999999999998,"15min":20.114999999999998},"min":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":167.67599999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":10.35}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.559999999999999,"5min":27.559999999999999,"15min":27.559999999999999},"min":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":215.80500000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":21.260000000000002},{"interface":"front","average":{"1min":27.331,"5min":27.331,"15min":27.331},"min":{"1min":8.6989999999999998,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":187.863,"5min":187.863,"15min":187.863},"last":14.045}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.262,"5min":27.262,"15min":27.262},"min":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":228.81200000000001,"5min":228.81200000000001,"15min":228.81200000000001},"last":11.069000000000001},{"interface":"front","average":{"1min":30.349,"5min":30.349,"15min":30.349},"min":{"1min":9.3710000000000004,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":275.92200000000003,"5min":275.92200000000003,"15min":275.92200000000003},"last":23.111000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.227,"5min":27.227,"15min":27.227},"min":{"1min":13.772,"5min":13.772,"15min":13.772},"max":{"1min":170.43600000000001,"5min":170.43600000000001,"15min":170.43600000000001},"last":24.042999999999999},{"interface":"front","average":{"1min":23.538,"5min":23.538,"15min":23.538},"min":{"1min":8.8360000000000003,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":157.00200000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":24.635000000000002}]}]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":131867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1130000000000004}]},{"osd":2,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.052,"5min":20.052,"15min":20.052},"min":{"1min":6.5110000000000001,"5min":6.5110000000000001,"15min":6.5110000000000001},"max":{"1min":127.95399999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":8.782},{"interface":"front","average":{"1min":14.872,"5min":14.872,"15min":14.872},"min":{"1min":7.5720000000000001,"5min":7.5720000000000001,"15min":7.5720000000000001},"max":{"1min":35.834000000000003,"5min":35.834000000000003,"15min":35.834000000000003},"last":24.652999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.649999999999999,"5min":22.649999999999999,"15min":22.649999999999999},"min":{"1min":8.984,"5min":8.984,"15min":8.984},"max":{"1min":99.790999999999997,"5min":99.790999999999997,"15min":99.790999999999997},"last":16.132000000000001},{"interface":"front","average":{"1min":16.715,"5min":16.715,"15min":16.715},"min":{"1min":9.4689999999999994,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":26.454999999999998,"5min":26.454999999999998,"15min":26.454999999999998},"last":26.454999999999998}]},{"osd":4,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":16.649999999999999,"5min":16.649999999999999,"15min":16.649999999999999},"min":{"1min":10.246,"5min":10.246,"15min":10.246},"max":{"1min":41.134999999999998,"5min":41.134999999999998,"15min":41.134999999999998},"last":16.640999999999998},{"interface":"front","average":{"1min":25.373999999999999,"5min":25.373999999999999,"15min":25.373999999999999},"min":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"max":{"1min":103.705,"5min":103.705,"15min":103.705},"last":18.891999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":19.039000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"min":{"1min":12.388999999999999,"5min":12.388999999999999,"15min":12.388999999999999},"max":{"1min":43.847000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":17.356999999999999},{"interface":"front","average":{"1min":28.347999999999999,"5min":28.347999999999999,"15min":28.347999999999999},"min":{"1min":12.076000000000001,"5min":12.076000000000001,"15min":12.076000000000001},"max":{"1min":98.707999999999998,"5min":98.707999999999998,"15min":98.707999999999998},"last":27.504000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.504000000000001,"5min":20.504000000000001,"15min":20.504000000000001},"min":{"1min":8.6869999999999994,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":43.192999999999998,"5min":43.192999999999998,"15min":43.192999999999998},"last":32.972999999999999},{"interface":"front","average":{"1min":22.300999999999998,"5min":22.300999999999998,"15min":22.300999999999998},"min":{"1min":13.728,"5min":13.728,"15min":13.728},"max":{"1min":44.545000000000002,"5min":44.545000000000002,"15min":44.545000000000002},"last":31.692}]},{"osd":7,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.123999999999999,"5min":22.123999999999999,"15min":22.123999999999999},"min":{"1min":11.917999999999999,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":45.927999999999997,"5min":45.927999999999997,"15min":45.927999999999997},"last":32.267000000000003},{"interface":"front","average":{"1min":23.283000000000001,"5min":23.283000000000001,"15min":23.283000000000001},"min":{"1min":13.298,"5min":13.298,"15min":13.298},"max":{"1min":45.130000000000003,"5min":45.130000000000003,"15min":45.130000000000003},"last":29.452000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":16.344999999999999,"5min":16.344999999999999,"15min":16.344999999999999},"min":{"1min":4.4249999999999998,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":131.477,"5min":131.477,"15min":131.477},"last":9.5519999999999996},{"interface":"front","average":{"1min":16.484000000000002,"5min":16.484000000000002,"15min":16.484000000000002},"min":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":129.386,"5min":129.386,"15min":129.386},"last":11.454000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":10.994,"5min":10.994,"15min":10.994},"min":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"max":{"1min":25.774000000000001,"5min":25.774000000000001,"15min":25.774000000000001},"last":6.5270000000000001},{"interface":"front","average":{"1min":19.489000000000001,"5min":19.489000000000001,"15min":19.489000000000001},"min":{"1min":9.9960000000000004,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":91.295000000000002,"5min":91.295000000000002,"15min":91.295000000000002},"last":16.585000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":18.219000000000001,"5min":18.219000000000001,"15min":18.219000000000001},"min":{"1min":6.7300000000000004,"5min":6.7300000000000004,"15min":6.7300000000000004},"max":{"1min":134.21199999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":7.4989999999999997},{"interface":"front","average":{"1min":14.635,"5min":14.635,"15min":14.635},"min":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"max":{"1min":27.780000000000001,"5min":27.780000000000001,"15min":27.780000000000001},"last":12.884}]},{"osd":3,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":20.669,"5min":20.669,"15min":20.669},"min":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"max":{"1min":132.56100000000001,"5min":132.56100000000001,"15min":132.56100000000001},"last":13.509},{"interface":"front","average":{"1min":21.187999999999999,"5min":21.187999999999999,"15min":21.187999999999999},"min":{"1min":6.5780000000000003,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":124.684,"5min":124.684,"15min":124.684},"last":18.065000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":13.888,"5min":13.888,"15min":13.888},"min":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":24.963000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":8.5579999999999998},{"interface":"front","average":{"1min":15.602,"5min":15.602,"15min":15.602},"min":{"1min":8.3089999999999993,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":24.84,"5min":24.84,"15min":24.84},"last":14.595000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.725000000000001,"5min":20.725000000000001,"15min":20.725000000000001},"min":{"1min":8.3179999999999996,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":125.792,"5min":125.792,"15min":125.792},"last":10.754},{"interface":"front","average":{"1min":17.007000000000001,"5min":17.007000000000001,"15min":17.007000000000001},"min":{"1min":10.273,"5min":10.273,"15min":10.273},"max":{"1min":28.591999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":15.852}]},{"osd":6,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":32.167999999999999,"5min":32.167999999999999,"15min":32.167999999999999},"min":{"1min":8.3480000000000008,"5min":8.3480000000000008,"15min":8.3480000000000008},"max":{"1min":327.29599999999999,"5min":327.29599999999999,"15min":327.29599999999999},"last":18.968},{"interface":"front","average":{"1min":33.918999999999997,"5min":33.918999999999997,"15min":33.918999999999997},"min":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"max":{"1min":323.82400000000001,"5min":323.82400000000001,"15min":323.82400000000001},"last":10.18}]}]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":540657,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":15.087,"5min":15.087,"15min":15.087},"min":{"1min":5.4829999999999997,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":68.513999999999996,"5min":68.513999999999996,"15min":68.513999999999996},"last":9.5190000000000001},{"interface":"front","average":{"1min":17.041,"5min":17.041,"15min":17.041},"min":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"max":{"1min":88.816000000000003,"5min":88.816000000000003,"15min":88.816000000000003},"last":10.311}]},{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":17.635000000000002,"5min":17.635000000000002,"15min":17.635000000000002},"min":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":87.515000000000001,"5min":87.515000000000001,"15min":87.515000000000001},"last":15.811},{"interface":"front","average":{"1min":18.013000000000002,"5min":18.013000000000002,"15min":18.013000000000002},"min":{"1min":6.21,"5min":6.21,"15min":6.21},"max":{"1min":85.832999999999998,"5min":85.832999999999998,"15min":85.832999999999998},"last":8.5410000000000004}]},{"osd":2,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":18.363,"5min":18.363,"15min":18.363},"min":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":113.943,"5min":113.943,"15min":113.943},"last":12.116},{"interface":"front","average":{"1min":15.470000000000001,"5min":15.470000000000001,"15min":15.470000000000001},"min":{"1min":6.5279999999999996,"5min":6.5279999999999996,"15min":6.5279999999999996},"max":{"1min":80.004000000000005,"5min":80.004000000000005,"15min":80.004000000000005},"last":17.050000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":21.841000000000001,"5min":21.841000000000001,"15min":21.841000000000001},"min":{"1min":7.3730000000000002,"5min":7.3730000000000002,"15min":7.3730000000000002},"max":{"1min":82.506,"5min":82.506,"15min":82.506},"last":7.7050000000000001},{"interface":"front","average":{"1min":23.952000000000002,"5min":23.952000000000002,"15min":23.952000000000002},"min":{"1min":9.6720000000000006,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":91.183000000000007,"5min":91.183000000000007,"15min":91.183000000000007},"last":11.526}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":12.738,"5min":12.738,"15min":12.738},"min":{"1min":7.0099999999999998,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":25.195,"5min":25.195,"15min":25.195},"last":18.167999999999999},{"interface":"front","average":{"1min":17.186,"5min":17.186,"15min":17.186},"min":{"1min":8.7100000000000009,"5min":8.7100000000000009,"15min":8.7100000000000009},"max":{"1min":59.609000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":12.839}]},{"osd":6,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":16.122,"5min":16.122,"15min":16.122},"min":{"1min":8.6820000000000004,"5min":8.6820000000000004,"15min":8.6820000000000004},"max":{"1min":66.765000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":13.971},{"interface":"front","average":{"1min":21.257999999999999,"5min":21.257999999999999,"15min":21.257999999999999},"min":{"1min":11.551,"5min":11.551,"15min":11.551},"max":{"1min":71.771000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":15.16}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.148,"5min":19.148,"15min":19.148},"min":{"1min":9.8770000000000007,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":64.677000000000007,"5min":64.677000000000007,"15min":64.677000000000007},"last":10.923},{"interface":"front","average":{"1min":18.381,"5min":18.381,"15min":18.381},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":42.531999999999996,"5min":42.531999999999996,"15min":42.531999999999996},"last":19.291}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":142817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"min":{"1min":4.5410000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":14.558,"5min":14.558,"15min":14.558},"last":9.8209999999999997},{"interface":"front","average":{"1min":10.558,"5min":10.558,"15min":10.558},"min":{"1min":6.2850000000000001,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":14.507999999999999,"5min":14.507999999999999,"15min":14.507999999999999},"last":8.8970000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":11.471,"5min":11.471,"15min":11.471},"min":{"1min":6.2960000000000003,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":19.382999999999999,"5min":19.382999999999999,"15min":19.382999999999999},"last":13.630000000000001},{"interface":"front","average":{"1min":14.153,"5min":14.153,"15min":14.153},"min":{"1min":6.609,"5min":6.609,"15min":6.609},"max":{"1min":26.864000000000001,"5min":26.864000000000001,"15min":26.864000000000001},"last":19.867999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.968,"5min":11.968,"15min":11.968},"min":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":16.492999999999999,"5min":16.492999999999999,"15min":16.492999999999999},"last":11.635},{"interface":"front","average":{"1min":15.776,"5min":15.776,"15min":15.776},"min":{"1min":12.058,"5min":12.058,"15min":12.058},"max":{"1min":21.231000000000002,"5min":21.231000000000002,"15min":21.231000000000002},"last":14.92}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.265000000000001,"5min":11.265000000000001,"15min":11.265000000000001},"min":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":29.881,"5min":29.881,"15min":29.881},"last":8.0600000000000005},{"interface":"front","average":{"1min":16.065999999999999,"5min":16.065999999999999,"15min":16.065999999999999},"min":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"max":{"1min":30.285,"5min":30.285,"15min":30.285},"last":12.34}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.603000000000002,"5min":19.603000000000002,"15min":19.603000000000002},"min":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":64.474999999999994,"5min":64.474999999999994,"15min":64.474999999999994},"last":21.122},{"interface":"front","average":{"1min":16.212,"5min":16.212,"15min":16.212},"min":{"1min":5.7770000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":75.980000000000004,"5min":75.980000000000004,"15min":75.980000000000004},"last":14.276}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.779,"5min":18.779,"15min":18.779},"min":{"1min":16.312000000000001,"5min":16.312000000000001,"15min":16.312000000000001},"max":{"1min":26.965,"5min":26.965,"15min":26.965},"last":22.251999999999999},{"interface":"front","average":{"1min":14.026,"5min":14.026,"15min":14.026},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":25.664000000000001,"5min":25.664000000000001,"15min":25.664000000000001},"last":18.143999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.358000000000001,"5min":18.358000000000001,"15min":18.358000000000001},"min":{"1min":3.5579999999999998,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":74.977999999999994,"5min":74.977999999999994,"15min":74.977999999999994},"last":15.619999999999999},{"interface":"front","average":{"1min":22.355,"5min":22.355,"15min":22.355},"min":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":83.138000000000005,"5min":83.138000000000005,"15min":83.138000000000005},"last":17.390999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:25:21.314 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [4, 2, 3, 7] 2023-10-30T06:25:21.314 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '4', '2', '3', '7'] 2023-10-30T06:25:21.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 4 2 3 7 2023-10-30T06:25:21.356 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.356 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:21.396 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:21.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:21.467 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.470 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:21.468+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:21.495 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.495 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.504 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:21.593 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:21.613 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.614 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:21.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:21.648 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:21.673 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.673 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:21.697 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:21.698 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.698 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:21.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:21.772 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:21.770+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:21.789 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:21.790 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.814 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:21.814 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:21.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:21.868 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:25:21.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:21.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.907 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:21.905+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:21.972 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:21.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:21.974 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:21.972+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:22.007 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:22.008 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:22.008 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:22.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:22.061 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:22.061 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:22.075 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:22.074+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:22.148 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:22.149 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:22.201 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:22.200+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:22.227 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:22.227 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:22.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:22.228 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:22.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.228 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:22.228 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:22.229 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 70.072028442000004, 2023-10-30T06:25:22.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:22.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:22.229 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.230 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:22.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:22.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:22.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:22.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:22.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:22.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:22.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:22.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:22.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:22.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:22.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:22.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:22.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.245 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:22.245 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:22.245 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 67.158854579000007, 2023-10-30T06:25:22.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:22.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.246 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:22.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:22.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:22.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:22.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:22.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:22.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:22.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:22.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:22.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:22.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:22.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:22.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:22.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.257 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:22.257 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:22.257 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.498676610000004, 2023-10-30T06:25:22.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:22.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.258 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:22.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:22.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:22.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:22.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:22.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:22.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:22.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:22.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:22.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:22.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:22.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:22.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:22.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:22.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:22.269 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:22.270 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.423253403000004, 2023-10-30T06:25:22.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:22.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:22.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:22.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:22.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:22.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:22.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:22.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:22.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:22.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:22.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:22.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:22.281 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.281 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.281 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.282 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:22.282 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:22.282 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.152130271999994, 2023-10-30T06:25:22.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:22.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.283 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:22.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:22.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:22.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:22.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:22.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:22.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:22.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:22.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:22.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:22.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:22.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:22.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:22.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:22.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:22.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:22.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:22.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.297 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:22.298 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.121850562000006, 2023-10-30T06:25:22.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:22.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.299 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:22.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:22.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:22.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:22.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:22.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:22.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:22.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.747939478000006, 2023-10-30T06:25:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:22.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.321 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.322 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:22.322 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:22.322 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.743440777000004, 2023-10-30T06:25:22.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:22.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.324 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:22.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:22.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:22.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:22.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:22.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:22.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:22.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:22.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:22.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:22.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:22.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:22.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:22.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:22.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:22.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:22.337 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.337 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.338 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:22.338 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.451836838000006, 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:22.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:22.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:22.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:22.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:22.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:22.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:22.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:22.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:22.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:22.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:22.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:22.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:22.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.349 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.350 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:22.350 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:22.350 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.445899521000001, 2023-10-30T06:25:22.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:22.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:22.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:22.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:22.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:22.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:22.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:22.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:22.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:22.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:22.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:22.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.362 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:22.362 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:22.362 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.444600506, 2023-10-30T06:25:22.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:22.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.363 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:22.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:22.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:22.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:22.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:22.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:22.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:22.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:22.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:22.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:22.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:22.373 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.374 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.374 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.341611697999994, 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.375 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:22.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:22.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:22.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:22.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:22.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:22.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:22.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:22.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:22.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:22.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:22.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:22.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.386 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.386 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:22.387 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:22.387 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.069708430999995, 2023-10-30T06:25:22.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:22.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:22.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:22.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:22.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:22.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:22.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:22.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:22.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:22.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:22.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:22.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:22.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:22.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:22.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:22.398 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.398 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.398 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.062210238999995, 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.400 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:22.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:22.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:22.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:22.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:22.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:22.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:22.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:22.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:22.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:22.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:22.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:22.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:22.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:22.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:22.411 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:22.412 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.061549033999995, 2023-10-30T06:25:22.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:22.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.412 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:22.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:22.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:22.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:22.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:22.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:22.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:22.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:22.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:22.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:22.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:22.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:22.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:22.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.423 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:22.424 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:22.424 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.048963196000003, 2023-10-30T06:25:22.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:22.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.425 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:22.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:22.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:22.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:22.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:22.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:22.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:22.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:22.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:22.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:22.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:22.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:22.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:22.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:22.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:22.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.435 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.436 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:22.436 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:22.437 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.042870983, 2023-10-30T06:25:22.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:22.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.437 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:22.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:22.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:22.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:22.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:22.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:22.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:22.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:22.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:22.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:22.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:22.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:22.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:22.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:22.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:22.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:22.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.451 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.452 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:22.452 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:22.452 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.041875641999994, 2023-10-30T06:25:22.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:22.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.453 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:22.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:22.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:22.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:22.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:22.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:22.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:22.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:22.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:22.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:22.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:22.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:22.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:22.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:22.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.465 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:22.465 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:22.465 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 57.719203071000003, 2023-10-30T06:25:22.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:22.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.466 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:22.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:22.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:22.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:22.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:22.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:22.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:22.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:22.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:22.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:22.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:22.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:22.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:22.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:22.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:22.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.477 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.478 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:22.478 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:22.478 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 57.685761585999998, 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:22.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:22.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:22.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:22.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:22.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:22.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:22.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:22.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:22.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:22.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:22.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:22.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:22.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:22.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:22.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:22.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:22.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:22.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:22.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:22.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:22.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:22.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:22.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:22.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:22.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:22.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:22.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:22.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:22.492 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:22.492 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:22.494 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:22.495 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:22.493+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:22.594 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:22.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:22.621 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:22.621 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:22.622 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:22.622 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:22.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:22.695 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:22.695+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:22.796 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:22.796+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:22.797 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:22.797 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:22.797 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:22.798 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:22.798 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:22.807 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:22.901 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:22.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:22.906 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.3 pg_upmap_items mapping to [4->2,3->7] 2023-10-30T06:25:22.963 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:22.963 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:22.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:22.964 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:22.964 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:22.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:23.015 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:23.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:23.099 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:23.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:23.144 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:23.144 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:23.144 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:23.144 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:23.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:23.199 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:23.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:23.300 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:23.299+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:23.323 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:23.323 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:23.323 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:23.324 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:23.324 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:23.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:23.401 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:23.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:23.502 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:23.502+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:23.504 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:23.504 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:23.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:23.504 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:23.504 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:23.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:23.603 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:23.603+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:23.645 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:23.646 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:23.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:23.740 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:23.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:23.805 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:23.805+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:23.822 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:23.823 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:23.823 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:23.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:23.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:25:23.917 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:23.916+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:24.007 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:24.007+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:24.035 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:24.036 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:24.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:24.036 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:24.036 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:24.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:24.108 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:24.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:24.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:24.209 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:24.207+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:24.214 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:24.214 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:24.214 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:24.214 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:24.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:24.310 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:24.309+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:24.358 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:24.359 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:24.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:24.411 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:24.410+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:24.476 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:24.476 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:24.512 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:24.510+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:24.566 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:24.566 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:24.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:24.567 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:24.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:24.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:24.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.228532639999997, 2023-10-30T06:25:24.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:24.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.569 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:24.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:24.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:24.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:24.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:24.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:24.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:24.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:24.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:24.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:24.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:24.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:24.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:24.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:24.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:24.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:24.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:24.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:24.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:24.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:24.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:24.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:24.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:24.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.584 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.585 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.585 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:24.585 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.907377448999995, 2023-10-30T06:25:24.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:24.585 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:24.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.587 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:24.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:24.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:24.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:24.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:24.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:24.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:24.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:24.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:24.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:24.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:24.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:24.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:24.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:24.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:24.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:24.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:24.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:24.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:24.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.600 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.600 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.600 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.903132892000002, 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.602 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.602 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:24.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.602 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:24.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:24.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:24.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:24.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:24.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:24.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:24.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:24.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:24.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:24.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:24.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:24.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:24.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:24.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:24.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:24.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:24.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.616 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.616 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:24.616 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.817674467000003, 2023-10-30T06:25:24.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:24.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.617 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:24.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.618 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:24.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:24.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:24.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:24.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:24.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:24.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:24.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:24.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:24.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:24.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:24.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:24.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:24.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:24.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:24.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:24.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:24.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:24.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:24.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:24.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:24.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.631 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.548431250999997, 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:24.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:24.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:24.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:24.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:24.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:24.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:24.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:24.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:24.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:24.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:24.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:24.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:24.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:24.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:24.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:24.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:24.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:24.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:24.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.547599840000004, 2023-10-30T06:25:24.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:24.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:24.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:24.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:24.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:24.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:24.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:24.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:24.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:24.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:24.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:24.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:24.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:24.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.546932713000004, 2023-10-30T06:25:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:24.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:24.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.524658642000006, 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:24.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:24.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:24.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:24.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:24.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:24.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:24.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:24.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:24.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:24.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:24.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:24.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.524010786000005, 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:24.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:24.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:24.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:24.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:24.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:24.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:24.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:24.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:24.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:24.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:24.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:24.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:24.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:24.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:24.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:24.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:24.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:24.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:24.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:24.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:24.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:24.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:24.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:24.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 68.523324712999994, 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:24.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:24.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:24.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:24.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:24.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:24.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:24.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:24.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:24.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:24.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:24.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:24.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:24.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:24.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:24.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:24.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:24.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.452442744999999, 2023-10-30T06:25:24.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:24.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:24.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:24.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:24.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:24.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:24.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:24.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:24.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:24.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:24.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:24.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:24.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:24.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:24.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:24.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:24.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:24.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:24.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:24.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:24.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:24.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:24.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:24.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:24.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.451778778999994, 2023-10-30T06:25:24.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:24.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:24.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:24.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:24.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:24.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:24.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:24.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:24.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:24.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:24.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:24.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:24.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:24.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:24.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:24.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:24.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:24.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:24.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:24.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:24.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:24.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 67.423324969999996, 2023-10-30T06:25:24.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:24.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:24.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:24.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:24.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:24.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:24.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:24.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:24.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:24.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:24.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:24.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:24.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:24.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:24.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:24.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:24.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:24.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:24.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:24.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:24.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:24.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:24.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.081558477000002, 2023-10-30T06:25:24.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:24.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:24.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:24.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:24.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:24.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:24.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:24.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:24.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:24.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:24.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:24.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:24.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:24.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:24.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:24.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:24.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:24.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:24.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:24.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:24.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:24.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:24.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:24.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:24.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.080809807000001, 2023-10-30T06:25:24.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:24.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:24.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:24.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:24.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:24.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:24.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:24.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:24.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:24.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:24.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:24.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:24.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:24.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:24.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:24.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:24.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:24.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:24.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:24.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:24.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:24.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:24.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.080141423999997, 2023-10-30T06:25:24.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:24.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:24.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:24.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:24.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:24.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:24.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:24.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:24.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:24.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:24.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:24.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:24.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:24.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.070290749999998, 2023-10-30T06:25:24.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:24.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:24.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:24.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:24.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.069616934999999, 2023-10-30T06:25:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:24.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:24.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:24.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:24.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:24.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:24.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:24.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:24.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:24.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:24.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:24.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.068968552000001, 2023-10-30T06:25:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:24.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:24.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:24.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:24.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:24.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:24.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:24.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 60.064852086000002, 2023-10-30T06:25:24.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:24.870 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:24.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:24.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:24.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:24.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:24.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:24.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:24.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:24.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:24.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.883 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:24.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:24.884 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:24.884 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:24.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:24.886 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:24.887 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:24.887 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:24.888 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:24.888 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:24.888 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:24.889 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:24.885+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:24.930 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:24.930 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:24.986 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:24.985+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:25.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:25.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:25.070 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.070 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:25.118 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:25.119 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:25.120 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:25.118+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:25.187 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:25.187+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:25.202 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:25.203 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.250 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:25.288 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:25.286+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:25.389 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:25.388+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:25.432 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.433 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:25.490 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:25.489+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:25.569 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:25.570 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.570 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:25.583 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:25:25.583 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T06:25:25.584 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:25:25.626 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:25.625+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:25.639 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:25:25.628+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 85991) UID: 0 2023-10-30T06:25:25.640 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:25:25.629+0000 9f64640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:25:25.692 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:25.691+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:25.784 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.785 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.792 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:25.792+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:25.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:25.894 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:25.893+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:25.967 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:25.968 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:25.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:25.994 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:25.993+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:26.095 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:26.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:26.104 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.105 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:26.196 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:26.196+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:26.208 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:21.410 86024== 2023-10-30T06:25:26.208 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:21.410 86024== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:25:26.218 INFO:tasks.ceph.mon.c.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:25:26.280 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.280 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:26.297 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:26.296+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:26.398 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:26.398+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:26.417 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.417 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:26.509 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:26.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:26.596 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.596 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.600 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:26.599+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:26.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:26.706 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:26.705+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:26.771 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.772 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:26.838 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:26.836+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:26.911 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:26.910+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:26.916 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:26.916 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:26.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:27.013 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:27.012+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:27.094 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.095 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:27.149 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:27.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:27.207 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:27.206+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:27.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:27.275 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.276 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:27.310 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:27.308+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:27.412 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:27.411+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:27.466 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:27.466 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.469 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.469 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:27.528 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:27.527+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:27.553 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:27.553 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.608 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.609 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:27.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:27.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.635 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:27.632+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:27.721 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:27.721 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.723 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:27.721+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:27.791 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.791 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:27.808 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:27.808 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.813 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:27.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:27.894 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:27.895 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:27.918 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:27.917+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:27.921 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:27.922 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:27.922 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2023-10-30T06:25:27.922 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:25:27.923 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:25:27.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2023-10-30T06:25:27.931 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:27.931 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:27.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:27.980 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:27.980 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:28.014 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:28.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:28.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:28.082 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.083 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2023-10-30T06:25:28.083 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.114 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.115 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:28.115 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:28.115 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.116 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:28.115+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:28.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:28.216 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:28.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:28.290 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.290 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:28.291 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:28.291 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:28.317 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:28.317+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:28.418 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:28.418+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:28.424 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:28.425 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:28.519 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:28.517+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:28.561 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.561 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:28.562 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:28.562 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:28.620 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:28.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:28.696 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:28.696 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:28.697 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:28.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:28.746 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:28.744+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:28.821 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:28.820+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:28.922 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:28.921+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:28.931 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:28.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:28.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 75.594278583999994, 2023-10-30T06:25:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:28.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:28.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:28.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:28.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:28.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:28.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:28.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:28.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:28.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:28.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:28.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:28.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:28.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:28.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.273123393000006, 2023-10-30T06:25:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:28.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:28.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:28.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:28.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:28.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:28.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:28.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:28.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:28.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:28.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:28.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:28.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.967 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:28.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.268878835999999, 2023-10-30T06:25:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:28.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:28.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:28.970 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:28.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.970 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:28.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:28.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:28.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:28.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:28.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:28.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:28.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:28.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:28.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:28.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:28.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:28.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:28.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:28.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:28.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:28.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:28.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:28.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:28.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:28.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:28.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:28.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:28.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:28.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:28.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:28.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:28.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:28.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:28.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:28.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:28.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:28.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.984 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:28.984 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:28.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.985 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:28.985 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:28.985 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.183420411, 2023-10-30T06:25:28.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:28.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:28.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.987 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:28.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:28.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:28.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:28.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:28.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:28.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:28.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:28.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:28.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:28.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:28.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:28.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:28.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:28.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:28.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:28.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:28.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:28.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:28.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:28.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:28.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:28.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:28.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:28.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:28.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:28.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:28.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:28.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:28.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:28.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:28.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:29.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:29.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.001 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.914177194999994, 2023-10-30T06:25:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.002 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.003 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.003 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:29.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.004 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:29.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:29.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:29.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:29.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:29.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:29.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:29.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:29.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:29.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:29.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:29.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:29.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:29.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:29.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:29.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:29.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:29.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:29.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:29.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.016 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.017 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.017 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.018 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:29.018 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.913345784000001, 2023-10-30T06:25:29.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:29.018 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.019 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.019 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.019 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:29.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:29.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:29.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:29.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:29.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:29.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:29.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:29.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:29.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:29.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:29.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:29.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:29.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.032 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.033 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.912678657000001, 2023-10-30T06:25:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.035 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:29.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:29.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.890404586000002, 2023-10-30T06:25:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:29.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:29.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:29.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:29.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.889756730000002, 2023-10-30T06:25:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:29.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:29.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:29.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:29.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:29.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 72.889070657000005, 2023-10-30T06:25:29.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:29.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:29.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:29.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:29.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.818188688999996, 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:29.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:29.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:29.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:29.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.817524723000005, 2023-10-30T06:25:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:29.112 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:29.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:29.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:29.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:29.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:29.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:29.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:29.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:29.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:29.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:29.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:29.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:29.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:29.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:29.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:29.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:29.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:29.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:29.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:29.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 71.789070914000007, 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.128 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:29.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.130 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:29.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:29.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:29.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:29.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:29.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:29.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:29.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:29.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:29.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:29.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:29.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:29.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:29.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:29.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:29.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:29.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.142 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.143 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.143 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:29.143 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.447304420999998, 2023-10-30T06:25:29.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.144 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:29.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.145 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:29.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:29.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:29.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:29.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:29.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:29.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:29.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:29.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:29.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:29.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:29.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:29.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:29.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:29.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:29.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:29.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:29.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:29.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:29.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.159 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:29.159 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.446555751000005, 2023-10-30T06:25:29.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:29.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.159 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.160 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:29.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:29.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:29.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:29.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:29.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:29.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:29.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:29.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:29.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:29.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:29.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:29.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:29.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:29.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.174 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.174 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:29.174 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.445887368000001, 2023-10-30T06:25:29.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:29.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.175 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:29.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:29.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:29.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:29.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:29.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:29.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:29.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:29.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:29.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:29.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:29.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:29.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:29.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:29.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:29.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.436036693999995, 2023-10-30T06:25:29.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:29.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.190 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:29.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:29.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:29.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:29.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:29.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:29.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:29.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:29.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:29.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:29.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:29.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:29.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:29.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:29.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:29.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:29.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:29.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:29.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:29.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:29.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.435362878999996, 2023-10-30T06:25:29.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:29.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.208 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:29.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.208 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:29.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:29.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:29.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:29.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:29.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:29.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:29.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:29.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:29.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:29.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:29.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:29.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:29.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:29.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:29.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:29.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:29.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.434714495999998, 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.223 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:29.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:29.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:29.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:29.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:29.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:29.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:29.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:29.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:29.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:29.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:29.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:29.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:29.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:29.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:29.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:29.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:29.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:29.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:29.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:29.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 64.430598029999999, 2023-10-30T06:25:29.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:29.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:29.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:29.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:29.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:29.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:29.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:29.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:29.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:29.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:29.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:29.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:29.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:29.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:29.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:29.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:29.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:29.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:29.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:29.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:29.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:29.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:29.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:29.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:29.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:29.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:29.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:29.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:29.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:29.254 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:29.254 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:29.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:29.257 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:29.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:29.356 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:29.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:29.424 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:29.424 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:29.425 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:29.425 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:29.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:29.478 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:29.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:29.558 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:29.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:29.560 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:29.560 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:29.560 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:29.560 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:29.561 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:29.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:29.658 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:29.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:29.736 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:29.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:29.737 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:29.737 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:29.746 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:29.786 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:29.785+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:29.860 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:29.858+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:29.911 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:29.912 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:29.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:29.961 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:29.960+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:30.063 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:30.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:30.086 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:30.087 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:30.087 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:30.087 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:30.087 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:30.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:30.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:30.164 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:30.163+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:30.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-30T06:25:30.264 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:30.265 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:30.265 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:30.266 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:30.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:30.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:30.365 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:30.363+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:30.407 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:30.407 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:30.407 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:30.408 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:30.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:30.466 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:30.464+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:30.546 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:30.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:30.546 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:30.546 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:30.547 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:30.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:30.596 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:30.594+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:30.667 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:30.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:30.720 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:30.721 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:30.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:30.721 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:30.721 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:30.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:30.771 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:30.770+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:30.869 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:30.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:30.897 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:30.897 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:30.897 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:30.897 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:30.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:30.969 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:30.968+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:31.070 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:31.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:31.073 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:31.073 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:31.073 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:31.073 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:31.073 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:31.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:31.173 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:31.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:31.289 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:31.289 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:31.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:31.290 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:31.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.290 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:31.290 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:31.291 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.133936386000002, 2023-10-30T06:25:31.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:31.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:31.291 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:31.292 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:31.292 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:31.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:31.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:31.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:31.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:31.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:31.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:31.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:31.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:31.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:31.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:31.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:31.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:31.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:31.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:31.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:31.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:31.303 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.303 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.303 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 76.220762523000005, 2023-10-30T06:25:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:31.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.305 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:31.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:31.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:31.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:31.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:31.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:31.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:31.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:31.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:31.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:31.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:31.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:31.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:31.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:31.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:31.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:31.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.315 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.316 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:31.316 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:31.316 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.560584554000002, 2023-10-30T06:25:31.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:31.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.317 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:31.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:31.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:31.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:31.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:31.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:31.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:31.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:31.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:31.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:31.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:31.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:31.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:31.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:31.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:31.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:31.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:31.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:31.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.327 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.328 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:31.328 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:31.328 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.485161347000002, 2023-10-30T06:25:31.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:31.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.329 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:31.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:31.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:31.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:31.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:31.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:31.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:31.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:31.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:31.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:31.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:31.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.340 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:31.340 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:31.340 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.214038216000006, 2023-10-30T06:25:31.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:31.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:31.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:31.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:31.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:31.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:31.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:31.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:31.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:31.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:31.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:31.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:31.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:31.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:31.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:31.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:31.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.351 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.183758506000004, 2023-10-30T06:25:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:31.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.353 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:31.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:31.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:31.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:31.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:31.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:31.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:31.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:31.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:31.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:31.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:31.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:31.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:31.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:31.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.363 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.364 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:31.364 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:31.364 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.809847422000004, 2023-10-30T06:25:31.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:31.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:31.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:31.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:31.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:31.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:31.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:31.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:31.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:31.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:31.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:31.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:31.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:31.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:31.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:31.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:31.374 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.375 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.375 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:31.376 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:31.376 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.805348721000001, 2023-10-30T06:25:31.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:31.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:31.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:31.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:31.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:31.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:31.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:31.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:31.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:31.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:31.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.387 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:31.388 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:31.389 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.513744782000003, 2023-10-30T06:25:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.507807464999999, 2023-10-30T06:25:31.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.506508449999998, 2023-10-30T06:25:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:31.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:31.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:31.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:31.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:31.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:31.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:31.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:31.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.403519642000006, 2023-10-30T06:25:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:31.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:31.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:31.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:31.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:31.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:31.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:31.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:31.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:31.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.131616374999993, 2023-10-30T06:25:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:31.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:31.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:31.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:31.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:31.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:31.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:31.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:31.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:31.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:31.448 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.124118182999993, 2023-10-30T06:25:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.449 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:31.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:31.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:31.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:31.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.459 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:31.460 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.123456977999993, 2023-10-30T06:25:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.461 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.471 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:31.472 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.11087114, 2023-10-30T06:25:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.104778926999998, 2023-10-30T06:25:31.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:31.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:31.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:31.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:31.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 74.103783586000006, 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:31.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:31.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:31.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:31.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:31.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:31.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:31.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:31.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:31.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:31.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.507 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.781111014999993, 2023-10-30T06:25:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:31.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:31.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:31.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:31.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:31.519 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 66.747669529999996, 2023-10-30T06:25:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:31.520 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:31.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:31.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:31.531 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:31.531 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:31.531 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:31.287+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:31.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:31.534 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:31.533+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:31.635 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:31.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:31.686 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:25:31.686 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-30T06:25:31.687 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T06:25:31.687 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:25:31.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:31.702 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:31.702 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:31.703 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:31.703 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:31.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:31.736 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:31.734+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:31.837 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:31.835+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:31.878 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:31.878 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:31.879 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:31.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:31.938 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:31.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:32.012 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:32.012 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:32.013 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:32.013 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:32.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:32.040 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:32.038+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:32.141 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:32.140+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:32.187 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:32.188 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:32.188 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:32.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:32.241 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:32.240+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:32.343 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:32.341+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:32.365 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:32.366 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:32.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:32.405 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:32.459 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:32.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:32.500 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:32.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:32.501 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:32.501 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:32.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:32.545 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:32.543+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:32.635 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:32.636 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:32.636 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:32.636 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:32.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:32.646 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:32.644+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:32.757 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:32.756+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:32.771 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:32.771 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:32.772 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:32.772 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:32.772 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:32.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:32.865 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:32.863+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:32.949 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:32.950 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:32.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:32.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:32.950 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:32.951 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:32.948+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:32.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:33.050 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:33.049+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:33.094 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:33.094 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:33.124 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:33.125 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:33.125 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:33.125 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:33.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:33.151 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:33.150+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:33.253 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:33.251+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:33.298 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:33.299 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:33.299 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:33.299 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:33.299 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:33.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:33.393 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:33.391+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:33.454 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:33.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:33.533 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:33.534 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:33.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:33.534 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:33.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.535 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:33.535 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:33.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.197192013999995, 2023-10-30T06:25:33.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:33.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:33.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.537 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:33.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:33.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:33.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:33.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:33.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:33.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:33.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:33.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:33.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:33.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:33.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:33.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:33.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:33.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:33.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:33.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:33.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.876036823000007, 2023-10-30T06:25:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:33.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:33.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:33.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:33.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:33.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:33.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:33.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:33.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.567 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.871792266, 2023-10-30T06:25:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.569 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:33.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:33.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.786333841000001, 2023-10-30T06:25:33.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.584 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:33.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:33.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:33.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:33.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:33.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:33.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:33.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.598 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.517090624999994, 2023-10-30T06:25:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:33.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:33.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:33.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.516259214000002, 2023-10-30T06:25:33.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.515592087000002, 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:33.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.493318016000003, 2023-10-30T06:25:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.492670160000003, 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:33.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:33.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:33.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:33.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:33.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:33.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:33.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:33.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:33.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:33.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:33.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:33.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:33.678 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 77.491984087000006, 2023-10-30T06:25:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.680 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:33.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:33.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:33.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:33.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.421102118999997, 2023-10-30T06:25:33.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.697 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:33.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.420438153000006, 2023-10-30T06:25:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:33.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:33.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:33.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:33.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.391984343999994, 2023-10-30T06:25:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:33.731 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.732 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:33.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:33.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:33.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:33.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:33.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.745 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.050217850999999, 2023-10-30T06:25:33.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.747 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:33.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:33.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:33.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:33.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:33.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:33.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:33.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:33.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:33.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:33.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:33.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:33.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.049469181000006, 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:33.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:33.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:33.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:33.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:33.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.048800798000002, 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.038950123999996, 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.038276308999997, 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.037627925999999, 2023-10-30T06:25:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 69.03351146, 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:33.860 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:33.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:33.863 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:33.860+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:33.969 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:33.967+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:34.034 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:34.035 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:34.064 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:34.062+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:34.165 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:34.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:34.170 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.171 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:34.171 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:34.171 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:34.172 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:34.265 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:34.264+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:34.307 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.307 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:34.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:34.308 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:34.308 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:34.367 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:34.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:34.468 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:34.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:34.485 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:34.486 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:34.569 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:34.567+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:34.623 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:34.624 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:34.670 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:34.669+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:34.771 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:34.770+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:34.799 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:34.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:34.800 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:34.800 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:34.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:34.873 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:34.871+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:34.951 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:34.952 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:34.952 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:34.961 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:35.003 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:35.001+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:35.074 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:35.073+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:35.137 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:35.137 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:35.137 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:35.138 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:35.138 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:35.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:35.187 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:35.186+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:35.277 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:35.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:35.313 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:35.313 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:35.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:35.314 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:35.314 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:35.323 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:35.407 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:35.406+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:35.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:35.480 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:35.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:35.494 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:35.495 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:35.495 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:35.495 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:35.504 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:35.579 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:35.578+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:35.669 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:35.670 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:35.670 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:35.670 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:35.670 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:35.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:35.681 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:35.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:35.782 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:35.780+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:35.845 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:35.845 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:35.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:35.846 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:35.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.846 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.689726368999999, 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:35.847 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:35.848 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:35.848 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:35.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:35.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.848 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:35.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:35.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:35.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:35.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:35.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:35.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:35.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:35.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:35.857 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.858 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.858 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.858 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 80.776552506000002, 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.859 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:35.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:35.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:35.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:35.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:35.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:35.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:35.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:35.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:35.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:35.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:35.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.870 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:35.870 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:35.870 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 80.116374536999999, 2023-10-30T06:25:35.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:35.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:35.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:35.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:35.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:35.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:35.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:35.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:35.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:35.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:35.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:35.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:35.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:35.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:35.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.880 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 80.040951329999999, 2023-10-30T06:25:35.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:35.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.882 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:35.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:35.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:35.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:35.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:35.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:35.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:35.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:35.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:35.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:35.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.891 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:35.892 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.769828199000003, 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:35.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:35.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:35.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:35.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:35.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:35.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:35.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:35.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:35.902 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.902 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.902 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.739548489000001, 2023-10-30T06:25:35.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:35.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.904 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:35.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:35.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:35.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:35.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:35.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:35.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:35.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:35.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:35.913 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.913 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:35.914 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.365637405000001, 2023-10-30T06:25:35.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:35.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.915 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:35.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:35.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:35.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:35.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:35.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:35.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:35.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:35.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:35.924 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.924 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:35.925 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.361138703999998, 2023-10-30T06:25:35.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:35.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.926 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:35.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:35.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:35.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:35.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:35.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:35.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:35.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:35.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:35.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.935 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.936 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:35.936 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.069534765, 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:35.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:35.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:35.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:35.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:35.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:35.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:35.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:35.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.946 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:35.947 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.063597447999996, 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:35.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:35.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:35.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:35.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.958 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.958 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 79.062298432999995, 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.959 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:35.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:35.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:35.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:35.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:35.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.966 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:35.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:35.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:35.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.970 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:35.970 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:35.970 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.959309625000003, 2023-10-30T06:25:35.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:35.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.971 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:35.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:35.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:35.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:35.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:35.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:35.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:35.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:35.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:35.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:35.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:35.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.981 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:35.981 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:35.981 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.687406358000004, 2023-10-30T06:25:35.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:35.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.982 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.982 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:35.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:35.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:35.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:35.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:35.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:35.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:35.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:35.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:35.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.991 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:35.992 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:35.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.992 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:35.992 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.679908166000004, 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:35.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:35.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:35.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:35.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:35.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:35.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:35.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:35.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:35.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:36.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:36.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:36.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:36.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:36.002 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:36.003 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:36.004 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.679246961000004, 2023-10-30T06:25:36.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:36.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.004 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:36.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:36.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:36.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:36.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:36.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:36.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:36.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:36.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:36.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.014 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:36.015 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:36.015 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.666661122999997, 2023-10-30T06:25:36.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:36.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:36.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:36.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:36.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:36.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:36.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:36.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:36.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.025 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:36.026 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:36.026 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.660568909999995, 2023-10-30T06:25:36.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:36.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:36.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:36.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:36.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:36.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:36.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:36.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:36.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:36.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:36.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.036 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 78.659573569000003, 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.038 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:36.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:36.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:36.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:36.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:36.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 71.336900998000004, 2023-10-30T06:25:36.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:36.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:36.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:36.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:36.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:36.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:36.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:36.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:36.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:36.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:36.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:36.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.058 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.059 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.059 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:36.059 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 71.303459513000007, 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:36.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:36.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:36.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:36.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:36.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:36.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:36.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:36.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:36.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:36.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:36.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:36.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:36.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:36.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:36.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.069 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:36.070 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:36.070 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:36.072 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:36.070+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:36.172 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:36.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:36.200 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.200 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:36.200 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:36.200 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:36.273 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:36.272+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:36.375 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:36.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:36.375 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:36.376 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:36.475 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:36.474+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:36.508 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.508 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:36.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:36.509 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:36.509 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:36.578 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:36.577+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:36.644 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.644 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:36.644 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:36.645 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:36.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-30T06:25:36.698 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:36.696+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:36.783 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.783 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:36.783 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:36.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:36.783 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.784 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:36.782+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:36.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:36.885 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:36.883+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:36.956 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:36.957 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:36.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:36.957 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:36.957 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:36.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:36.986 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:36.985+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:37.088 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:37.086+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:37.094 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:37.095 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:37.095 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:37.095 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:37.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:37.190 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:37.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:37.269 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:37.270 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:37.270 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:37.270 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:37.271 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:37.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:37.323 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:37.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:37.395 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:37.393+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:37.406 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:37.406 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:37.407 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:37.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:37.498 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:37.497+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:37.582 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:37.583 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:37.583 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:37.591 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:37.633 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:37.631+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:37.699 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:37.698+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:37.768 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:37.769 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:37.769 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:37.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:37.821 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:37.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:37.903 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:37.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:38.002 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:38.002 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:38.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:38.003 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:38.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:38.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:38.004 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 84.665134640000005, 2023-10-30T06:25:38.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:38.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.005 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:38.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:38.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:38.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:38.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:38.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:38.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:38.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:38.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:38.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:38.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.343979449000003, 2023-10-30T06:25:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.027 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:38.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:38.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:38.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:38.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.339734891999996, 2023-10-30T06:25:38.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.042 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:38.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:38.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:38.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:38.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.055 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:38.056 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 82.254276466999997, 2023-10-30T06:25:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:38.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:38.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:38.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:38.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.985033251000004, 2023-10-30T06:25:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:38.071 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:38.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:38.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:38.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:38.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:38.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.984201839999997, 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:38.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:38.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:38.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.983534712999997, 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.102 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:38.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:38.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:38.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:38.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:38.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:38.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:38.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:38.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:38.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:38.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:38.116 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.961260641999999, 2023-10-30T06:25:38.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:38.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:38.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:38.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:38.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:38.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:38.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:38.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:38.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:38.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:38.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:38.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:38.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:38.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:38.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:38.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:38.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:38.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:38.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:38.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:38.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.131 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.131 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:38.131 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.960612785999999, 2023-10-30T06:25:38.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:38.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.132 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.132 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:38.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:38.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:38.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:38.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:38.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:38.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:38.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:38.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:38.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:38.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:38.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:38.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:38.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:38.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:38.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:38.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.145 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:38.146 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:38.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 81.959926713000002, 2023-10-30T06:25:38.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:38.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.147 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:38.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:38.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:38.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:38.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:38.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:38.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:38.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:38.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:38.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:38.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:38.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:38.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:38.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:38.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.889044745000007, 2023-10-30T06:25:38.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:38.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.162 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:38.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:38.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:38.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:38.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:38.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:38.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:38.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:38.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:38.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.888380779000002, 2023-10-30T06:25:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:38.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:38.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:38.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:38.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.191 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.191 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 80.859926970000004, 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:38.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:38.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:38.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:38.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:38.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:38.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:38.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:38.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:38.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:38.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:38.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:38.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:38.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:38.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:38.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:38.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.518160476999995, 2023-10-30T06:25:38.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:38.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.210 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:38.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:38.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:38.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:38.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:38.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:38.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.517411807000002, 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:38.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:38.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:38.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:38.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:38.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:38.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:38.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.516743423999998, 2023-10-30T06:25:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:38.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:38.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:38.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:38.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:38.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.253 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.506892750000006, 2023-10-30T06:25:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:38.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:38.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.506218935000007, 2023-10-30T06:25:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:38.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:38.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:38.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.505570551999995, 2023-10-30T06:25:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:38.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:38.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:38.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:38.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 73.501454085999995, 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:38.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:38.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:38.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:38.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:38.316 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:38.317 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:38.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:38.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:25:38.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:38.319 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:38.316+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:38.419 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:38.417+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:38.488 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:38.489 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:38.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:38.520 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:38.519+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:38.621 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:38.619+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:38.625 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:38.626 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:38.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:38.722 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:38.721+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:38.805 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:38.806 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:38.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:38.806 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:38.806 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:38.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:38.823 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:38.822+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:38.924 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:38.923+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:38.981 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:38.981 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:38.982 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:38.982 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:38.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:39.035 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:39.033+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:39.118 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:39.118 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:39.119 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:39.119 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:39.126 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:39.124+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:39.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:39.227 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:39.225+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:39.305 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:39.305 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:39.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:39.305 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:25:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 3.0032769680000002, 2023-10-30T06:25:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:25:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:25:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:25:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:39.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:39.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:39.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:39.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:39.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:39.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:39.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:25:39.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:25:39.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:39.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:25:39.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:25:39.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:39.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:25:39.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:25:39.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:39.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:25:39.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:25:39.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:39.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:39.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:39.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:25:39.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:39.316 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:39.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:39.363 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:39.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:39.429 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:39.427+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:39.455 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:39.456 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:39.456 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:39.456 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:39.465 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:39.549 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:39.548+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:39.630 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:39.631 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:39.631 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:39.631 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:39.631 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:39.632 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:39.629+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:39.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:39.731 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:39.730+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:39.814 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:39.815 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:39.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:39.867 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:39.866+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:39.932 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:39.931+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:39.950 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:39.951 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:39.951 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:39.951 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:39.959 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:40.046 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:40.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:40.129 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:40.129 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:40.130 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:40.130 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:40.130 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:40.133 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:40.132+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:40.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:40.234 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:40.233+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:40.335 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:40.334+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:40.350 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:40.350 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:40.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:40.351 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:40.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.351 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:40.351 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:40.351 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.192295767000005, 2023-10-30T06:25:40.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:40.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:40.352 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:40.352 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:40.353 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:40.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:40.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.353 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:40.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:40.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:40.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:40.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:40.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:40.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:40.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:40.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:40.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:40.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:40.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:40.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:40.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:40.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:40.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.363 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:40.364 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 85.279121903999993, 2023-10-30T06:25:40.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:40.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.365 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:40.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:40.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:40.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:40.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:40.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:40.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:40.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:40.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:40.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.376 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:40.376 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:40.376 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 84.618943935000004, 2023-10-30T06:25:40.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:40.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:40.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:40.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:40.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:40.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:40.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:40.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:40.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:40.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:40.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:40.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:40.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:40.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:40.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:40.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.386 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.387 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:40.387 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:40.388 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 84.543520728000004, 2023-10-30T06:25:40.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:40.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.388 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:40.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:40.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:40.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:40.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:40.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:40.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:40.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:40.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:40.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:40.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:40.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:40.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:40.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:40.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.400 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:40.400 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:40.400 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 84.272397596999994, 2023-10-30T06:25:40.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:40.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.401 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:40.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:40.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:40.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:40.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:40.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:40.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:40.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:40.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:40.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:40.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:40.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:40.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:40.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:40.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.411 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.412 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:40.412 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:40.412 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 84.242117887000006, 2023-10-30T06:25:40.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:40.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:40.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:40.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:40.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:40.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:40.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:40.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:40.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:40.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:40.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:40.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:40.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:40.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:40.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:40.422 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.422 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:40.423 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.868206803000007, 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:40.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:40.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:40.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:40.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:40.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:40.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:40.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:40.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:40.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:40.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:40.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:40.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:40.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:40.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.435 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.436 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:40.436 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:40.436 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.863708102000004, 2023-10-30T06:25:40.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:40.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.437 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:40.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:40.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:40.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:40.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:40.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:40.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:40.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:40.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:40.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:40.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:40.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.447 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.448 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:40.448 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:40.448 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.572104163000006, 2023-10-30T06:25:40.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:40.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.449 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:40.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:40.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:40.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:40.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:40.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:40.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:40.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:40.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:40.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:40.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:40.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.458 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.459 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:40.459 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.566166846000002, 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:40.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:40.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:40.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:40.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:40.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:40.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:40.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.470 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:40.471 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:40.471 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.564867831000001, 2023-10-30T06:25:40.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:40.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.472 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:40.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:40.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:40.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:40.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:40.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:40.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:40.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.461879022999995, 2023-10-30T06:25:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.189975755999995, 2023-10-30T06:25:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:40.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:40.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.182477563999996, 2023-10-30T06:25:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.181816358999995, 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.169230521000003, 2023-10-30T06:25:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.163138308000001, 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 83.162142966999994, 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.839470395999996, 2023-10-30T06:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.806028910999999, 2023-10-30T06:25:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:40.584 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:40.585 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:40.586 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.587 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:40.587 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:40.588 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:40.584+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:40.591 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:40.591 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.687 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:40.685+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:40.703 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:40.703 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.737 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:40.738 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:25:40.751 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-30T06:25:40.176155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219480,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755240,"statfs":{"total":773094113280,"available":772869365760,"internally_reserved":0,"allocated":3383296,"data_stored":2070512,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.905883"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:23.006072+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:25:23.006072+0000","last_peered":"2023-10-30T06:25:23.006072+0000","last_clean":"2023-10-30T06:25:23.006072+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:25:23.006072+0000","last_undegraded":"2023-10-30T06:25:23.006072+0000","last_fullsized":"2023-10-30T06:25:23.006072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:24.260270+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:25:24.260270+0000","last_peered":"2023-10-30T06:25:24.260270+0000","last_clean":"2023-10-30T06:25:24.260270+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:25:24.260270+0000","last_undegraded":"2023-10-30T06:25:24.260270+0000","last_fullsized":"2023-10-30T06:25:24.260270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:24.258057+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:25:24.258057+0000","last_peered":"2023-10-30T06:25:24.258057+0000","last_clean":"2023-10-30T06:25:24.258057+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:25:24.258057+0000","last_undegraded":"2023-10-30T06:25:24.258057+0000","last_fullsized":"2023-10-30T06:25:24.258057+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:24.264642+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:25:24.264642+0000","last_peered":"2023-10-30T06:25:24.264642+0000","last_clean":"2023-10-30T06:25:24.264642+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:25:24.264642+0000","last_undegraded":"2023-10-30T06:25:24.264642+0000","last_fullsized":"2023-10-30T06:25:24.264642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:23.003125+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:25:23.003125+0000","last_peered":"2023-10-30T06:25:23.003125+0000","last_clean":"2023-10-30T06:25:23.003125+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:25:23.003125+0000","last_undegraded":"2023-10-30T06:25:23.003125+0000","last_fullsized":"2023-10-30T06:25:23.003125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:23.024843+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:25:23.024843+0000","last_peered":"2023-10-30T06:25:23.024843+0000","last_clean":"2023-10-30T06:25:23.024843+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:25:23.024843+0000","last_undegraded":"2023-10-30T06:25:23.024843+0000","last_fullsized":"2023-10-30T06:25:23.024843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:22.999244+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:25:22.999244+0000","last_peered":"2023-10-30T06:25:22.999244+0000","last_clean":"2023-10-30T06:25:22.999244+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:25:22.999244+0000","last_undegraded":"2023-10-30T06:25:22.999244+0000","last_fullsized":"2023-10-30T06:25:22.999244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-30T06:25:24.828593+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:25:24.828593+0000","last_peered":"2023-10-30T06:25:24.828593+0000","last_clean":"2023-10-30T06:25:24.828593+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:25:24.828593+0000","last_undegraded":"2023-10-30T06:25:24.828593+0000","last_fullsized":"2023-10-30T06:25:24.828593+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":131,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-30T06:25:24.256270+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:25:24.256270+0000","last_peered":"2023-10-30T06:25:24.256270+0000","last_clean":"2023-10-30T06:25:24.256270+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:25:24.256270+0000","last_undegraded":"2023-10-30T06:25:24.256270+0000","last_fullsized":"2023-10-30T06:25:24.256270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411349,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":557194,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.359999999999999,"5min":10.359999999999999,"15min":10.359999999999999},"min":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":24.350000000000001,"5min":24.350000000000001,"15min":24.350000000000001},"last":8.0030000000000001},{"interface":"front","average":{"1min":12.083,"5min":12.083,"15min":12.083},"min":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"max":{"1min":46.798000000000002,"5min":46.798000000000002,"15min":46.798000000000002},"last":14.286}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.798,"5min":13.798,"15min":13.798},"min":{"1min":5.5209999999999999,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":52.552,"5min":52.552,"15min":52.552},"last":10.239000000000001},{"interface":"front","average":{"1min":15.147,"5min":15.147,"15min":15.147},"min":{"1min":7.1589999999999998,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":27.946000000000002,"5min":27.946000000000002,"15min":27.946000000000002},"last":13.505000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"min":{"1min":11.013,"5min":11.013,"15min":11.013},"max":{"1min":118.45699999999999,"5min":118.45699999999999,"15min":118.45699999999999},"last":12.279},{"interface":"front","average":{"1min":22.571999999999999,"5min":22.571999999999999,"15min":22.571999999999999},"min":{"1min":7.8739999999999997,"5min":7.8739999999999997,"15min":7.8739999999999997},"max":{"1min":124.075,"5min":124.075,"15min":124.075},"last":14.891}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.675000000000001,"5min":13.675000000000001,"15min":13.675000000000001},"min":{"1min":7.2359999999999998,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":51.649000000000001,"5min":51.649000000000001,"15min":51.649000000000001},"last":18.245000000000001},{"interface":"front","average":{"1min":17.998999999999999,"5min":17.998999999999999,"15min":17.998999999999999},"min":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":38.271999999999998,"5min":38.271999999999998,"15min":38.271999999999998},"last":17.661999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.052,"5min":21.052,"15min":21.052},"min":{"1min":10.776,"5min":10.776,"15min":10.776},"max":{"1min":123.754,"5min":123.754,"15min":123.754},"last":15.470000000000001},{"interface":"front","average":{"1min":23.079999999999998,"5min":23.079999999999998,"15min":23.079999999999998},"min":{"1min":8.5679999999999996,"5min":8.5679999999999996,"15min":8.5679999999999996},"max":{"1min":122.72499999999999,"5min":122.72499999999999,"15min":122.72499999999999},"last":18.916}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.457000000000001,"5min":21.457000000000001,"15min":21.457000000000001},"min":{"1min":11.9,"5min":11.9,"15min":11.9},"max":{"1min":66.400999999999996,"5min":66.400999999999996,"15min":66.400999999999996},"last":20.981000000000002},{"interface":"front","average":{"1min":19.859000000000002,"5min":19.859000000000002,"15min":19.859000000000002},"min":{"1min":8.2070000000000007,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":53.149000000000001,"5min":53.149000000000001,"15min":53.149000000000001},"last":21.834}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.513999999999999,"5min":21.513999999999999,"15min":21.513999999999999},"min":{"1min":10.657,"5min":10.657,"15min":10.657},"max":{"1min":45.235999999999997,"5min":45.235999999999997,"15min":45.235999999999997},"last":16.129999999999999},{"interface":"front","average":{"1min":21.594999999999999,"5min":21.594999999999999,"15min":21.594999999999999},"min":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":69.991,"5min":69.991,"15min":69.991},"last":19.648}]}]},{"osd":6,"up_from":16,"seq":68719476758,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":14.706,"5min":14.706,"15min":14.706},"min":{"1min":4.8300000000000001,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":94.343000000000004,"5min":94.343000000000004,"15min":94.343000000000004},"last":15.677},{"interface":"front","average":{"1min":15.754,"5min":15.754,"15min":15.754},"min":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":122.054,"5min":122.054,"15min":122.054},"last":10.675000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"min":{"1min":6.4379999999999997,"5min":6.4379999999999997,"15min":6.4379999999999997},"max":{"1min":18.992999999999999,"5min":18.992999999999999,"15min":18.992999999999999},"last":8.0800000000000001},{"interface":"front","average":{"1min":33.863999999999997,"5min":33.863999999999997,"15min":33.863999999999997},"min":{"1min":5.9269999999999996,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":387.15600000000001,"5min":387.15600000000001,"15min":387.15600000000001},"last":17.724}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.956,"5min":19.956,"15min":19.956},"min":{"1min":6.4359999999999999,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":156.90299999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":9.5380000000000003},{"interface":"front","average":{"1min":20.584,"5min":20.584,"15min":20.584},"min":{"1min":8.8940000000000001,"5min":8.8940000000000001,"15min":8.8940000000000001},"max":{"1min":143.578,"5min":143.578,"15min":143.578},"last":18.510000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.963000000000001,"5min":21.963000000000001,"15min":21.963000000000001},"min":{"1min":10.696999999999999,"5min":10.696999999999999,"15min":10.696999999999999},"max":{"1min":141.18799999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":19.288},{"interface":"front","average":{"1min":25.420999999999999,"5min":25.420999999999999,"15min":25.420999999999999},"min":{"1min":13.539999999999999,"5min":13.539999999999999,"15min":13.539999999999999},"max":{"1min":161.32900000000001,"5min":161.32900000000001,"15min":161.32900000000001},"last":20.036000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":15.045,"5min":15.045,"15min":15.045},"min":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"max":{"1min":74.079999999999998,"5min":74.079999999999998,"15min":74.079999999999998},"last":13.763999999999999},{"interface":"front","average":{"1min":15.702,"5min":15.702,"15min":15.702},"min":{"1min":9.2270000000000003,"5min":9.2270000000000003,"15min":9.2270000000000003},"max":{"1min":58.613999999999997,"5min":58.613999999999997,"15min":58.613999999999997},"last":12.593999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":17.966000000000001,"5min":17.966000000000001,"15min":17.966000000000001},"min":{"1min":9.6560000000000006,"5min":9.6560000000000006,"15min":9.6560000000000006},"max":{"1min":71.757000000000005,"5min":71.757000000000005,"15min":71.757000000000005},"last":11.359},{"interface":"front","average":{"1min":21.762,"5min":21.762,"15min":21.762},"min":{"1min":14.734,"5min":14.734,"15min":14.734},"max":{"1min":80.119,"5min":80.119,"15min":80.119},"last":20.797000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.741,"5min":19.741,"15min":19.741},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":81.247,"5min":81.247,"15min":81.247},"last":16.324000000000002},{"interface":"front","average":{"1min":19.088000000000001,"5min":19.088000000000001,"15min":19.088000000000001},"min":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":70.989000000000004,"5min":70.989000000000004,"15min":70.989000000000004},"last":14.747999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":11.06,"5min":11.06,"15min":11.06},"min":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":24.466000000000001,"5min":24.466000000000001,"15min":24.466000000000001},"last":6.7729999999999997},{"interface":"front","average":{"1min":10.257999999999999,"5min":10.257999999999999,"15min":10.257999999999999},"min":{"1min":6,"5min":6,"15min":6},"max":{"1min":19.263999999999999,"5min":19.263999999999999,"15min":19.263999999999999},"last":13.536}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":26.827000000000002,"5min":26.827000000000002,"15min":26.827000000000002},"min":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":165.76400000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":7.4039999999999999},{"interface":"front","average":{"1min":28.079000000000001,"5min":28.079000000000001,"15min":28.079000000000001},"min":{"1min":8.5760000000000005,"5min":8.5760000000000005,"15min":8.5760000000000005},"max":{"1min":190.34200000000001,"5min":190.34200000000001,"15min":190.34200000000001},"last":8.5090000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":18.428000000000001,"5min":18.428000000000001,"15min":18.428000000000001},"min":{"1min":12.856999999999999,"5min":12.856999999999999,"15min":12.856999999999999},"max":{"1min":41.229999999999997,"5min":41.229999999999997,"15min":41.229999999999997},"last":15.800000000000001},{"interface":"front","average":{"1min":17.199000000000002,"5min":17.199000000000002,"15min":17.199000000000002},"min":{"1min":8.1739999999999995,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":33.060000000000002,"5min":33.060000000000002,"15min":33.060000000000002},"last":9.1639999999999997}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.620000000000001,"5min":20.620000000000001,"15min":20.620000000000001},"min":{"1min":9.9420000000000002,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":42.274999999999999,"5min":42.274999999999999,"15min":42.274999999999999},"last":16.640999999999998},{"interface":"front","average":{"1min":11.614000000000001,"5min":11.614000000000001,"15min":11.614000000000001},"min":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":12.394}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.658999999999999,"5min":19.658999999999999,"15min":19.658999999999999},"min":{"1min":7.4450000000000003,"5min":7.4450000000000003,"15min":7.4450000000000003},"max":{"1min":56.796999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":14.156000000000001},{"interface":"front","average":{"1min":20.798999999999999,"5min":20.798999999999999,"15min":20.798999999999999},"min":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"max":{"1min":58.576000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":9.8119999999999994}]},{"osd":6,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":29.138000000000002,"5min":29.138000000000002,"15min":29.138000000000002},"min":{"1min":11.694000000000001,"5min":11.694000000000001,"15min":11.694000000000001},"max":{"1min":169.05199999999999,"5min":169.05199999999999,"15min":169.05199999999999},"last":17.548999999999999},{"interface":"front","average":{"1min":15.494,"5min":15.494,"15min":15.494},"min":{"1min":9.6419999999999995,"5min":9.6419999999999995,"15min":9.6419999999999995},"max":{"1min":30.957000000000001,"5min":30.957000000000001,"15min":30.957000000000001},"last":11.787000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":17.986000000000001,"5min":17.986000000000001,"15min":17.986000000000001},"min":{"1min":8.5950000000000006,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":38.887999999999998,"5min":38.887999999999998,"15min":38.887999999999998},"last":10.404},{"interface":"front","average":{"1min":29.614999999999998,"5min":29.614999999999998,"15min":29.614999999999998},"min":{"1min":10.025,"5min":10.025,"15min":10.025},"max":{"1min":170.232,"5min":170.232,"15min":170.232},"last":18.332000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476756,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.809000000000001,"5min":27.809000000000001,"15min":27.809000000000001},"min":{"1min":5.0730000000000004,"5min":5.0730000000000004,"15min":5.0730000000000004},"max":{"1min":258.66199999999998,"5min":258.66199999999998,"15min":258.66199999999998},"last":16.039000000000001},{"interface":"front","average":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":203.38499999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":20.347999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":16.311,"5min":16.311,"15min":16.311},"min":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":45.936999999999998,"5min":45.936999999999998,"15min":45.936999999999998},"last":8.4489999999999998},{"interface":"front","average":{"1min":14.839,"5min":14.839,"15min":14.839},"min":{"1min":6.8360000000000003,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":37.390999999999998,"5min":37.390999999999998,"15min":37.390999999999998},"last":16.605}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.326000000000001,"5min":19.326000000000001,"15min":19.326000000000001},"min":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":183.38900000000001,"5min":183.38900000000001,"15min":183.38900000000001},"last":10.787000000000001},{"interface":"front","average":{"1min":23.067,"5min":23.067,"15min":23.067},"min":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":189.374,"5min":189.374,"15min":189.374},"last":12.226000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.959,"5min":21.959,"15min":21.959},"min":{"1min":7.7229999999999999,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":186.01400000000001,"5min":186.01400000000001,"15min":186.01400000000001},"last":17.215},{"interface":"front","average":{"1min":20.114999999999998,"5min":20.114999999999998,"15min":20.114999999999998},"min":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":167.67599999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":9.4269999999999996}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.559999999999999,"5min":27.559999999999999,"15min":27.559999999999999},"min":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":215.80500000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":21.547000000000001},{"interface":"front","average":{"1min":27.331,"5min":27.331,"15min":27.331},"min":{"1min":8.6989999999999998,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":187.863,"5min":187.863,"15min":187.863},"last":19.696999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.262,"5min":27.262,"15min":27.262},"min":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":228.81200000000001,"5min":228.81200000000001,"15min":228.81200000000001},"last":12.869999999999999},{"interface":"front","average":{"1min":30.349,"5min":30.349,"15min":30.349},"min":{"1min":9.3710000000000004,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":275.92200000000003,"5min":275.92200000000003,"15min":275.92200000000003},"last":22.196000000000002}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.227,"5min":27.227,"15min":27.227},"min":{"1min":13.772,"5min":13.772,"15min":13.772},"max":{"1min":170.43600000000001,"5min":170.43600000000001,"15min":170.43600000000001},"last":18.405999999999999},{"interface":"front","average":{"1min":23.538,"5min":23.538,"15min":23.538},"min":{"1min":8.8360000000000003,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":157.00200000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":13.449999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476756,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":10.695,"5min":10.695,"15min":10.695},"min":{"1min":6.109,"5min":6.109,"15min":6.109},"max":{"1min":25.989999999999998,"5min":25.989999999999998,"15min":25.989999999999998},"last":5.1790000000000003},{"interface":"front","average":{"1min":11.907999999999999,"5min":11.907999999999999,"15min":11.907999999999999},"min":{"1min":5.0010000000000003,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":30.788,"5min":30.788,"15min":30.788},"last":10.375}]},{"osd":2,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.052,"5min":20.052,"15min":20.052},"min":{"1min":6.5110000000000001,"5min":6.5110000000000001,"15min":6.5110000000000001},"max":{"1min":127.95399999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":9.9139999999999997},{"interface":"front","average":{"1min":14.872,"5min":14.872,"15min":14.872},"min":{"1min":7.5720000000000001,"5min":7.5720000000000001,"15min":7.5720000000000001},"max":{"1min":35.834000000000003,"5min":35.834000000000003,"15min":35.834000000000003},"last":9.0419999999999998}]},{"osd":3,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.649999999999999,"5min":22.649999999999999,"15min":22.649999999999999},"min":{"1min":8.984,"5min":8.984,"15min":8.984},"max":{"1min":99.790999999999997,"5min":99.790999999999997,"15min":99.790999999999997},"last":11.016},{"interface":"front","average":{"1min":16.715,"5min":16.715,"15min":16.715},"min":{"1min":9.4689999999999994,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":26.454999999999998,"5min":26.454999999999998,"15min":26.454999999999998},"last":9.6430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":16.649999999999999,"5min":16.649999999999999,"15min":16.649999999999999},"min":{"1min":10.246,"5min":10.246,"15min":10.246},"max":{"1min":41.134999999999998,"5min":41.134999999999998,"15min":41.134999999999998},"last":12.348000000000001},{"interface":"front","average":{"1min":25.373999999999999,"5min":25.373999999999999,"15min":25.373999999999999},"min":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"max":{"1min":103.705,"5min":103.705,"15min":103.705},"last":11.722}]},{"osd":5,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":19.039000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"min":{"1min":12.388999999999999,"5min":12.388999999999999,"15min":12.388999999999999},"max":{"1min":43.847000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":12.98},{"interface":"front","average":{"1min":28.347999999999999,"5min":28.347999999999999,"15min":28.347999999999999},"min":{"1min":12.076000000000001,"5min":12.076000000000001,"15min":12.076000000000001},"max":{"1min":98.707999999999998,"5min":98.707999999999998,"15min":98.707999999999998},"last":16.465}]},{"osd":6,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.504000000000001,"5min":20.504000000000001,"15min":20.504000000000001},"min":{"1min":8.6869999999999994,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":43.192999999999998,"5min":43.192999999999998,"15min":43.192999999999998},"last":13.548999999999999},{"interface":"front","average":{"1min":22.300999999999998,"5min":22.300999999999998,"15min":22.300999999999998},"min":{"1min":13.728,"5min":13.728,"15min":13.728},"max":{"1min":44.545000000000002,"5min":44.545000000000002,"15min":44.545000000000002},"last":17.417999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.123999999999999,"5min":22.123999999999999,"15min":22.123999999999999},"min":{"1min":11.917999999999999,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":45.927999999999997,"5min":45.927999999999997,"15min":45.927999999999997},"last":14.125999999999999},{"interface":"front","average":{"1min":23.283000000000001,"5min":23.283000000000001,"15min":23.283000000000001},"min":{"1min":13.298,"5min":13.298,"15min":13.298},"max":{"1min":45.130000000000003,"5min":45.130000000000003,"15min":45.130000000000003},"last":18.32}]}]},{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":16.344999999999999,"5min":16.344999999999999,"15min":16.344999999999999},"min":{"1min":4.4249999999999998,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":131.477,"5min":131.477,"15min":131.477},"last":6.29},{"interface":"front","average":{"1min":16.484000000000002,"5min":16.484000000000002,"15min":16.484000000000002},"min":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":129.386,"5min":129.386,"15min":129.386},"last":11.308}]},{"osd":1,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":10.994,"5min":10.994,"15min":10.994},"min":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"max":{"1min":25.774000000000001,"5min":25.774000000000001,"15min":25.774000000000001},"last":8.8659999999999997},{"interface":"front","average":{"1min":19.489000000000001,"5min":19.489000000000001,"15min":19.489000000000001},"min":{"1min":9.9960000000000004,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":91.295000000000002,"5min":91.295000000000002,"15min":91.295000000000002},"last":16.661999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":18.219000000000001,"5min":18.219000000000001,"15min":18.219000000000001},"min":{"1min":6.7300000000000004,"5min":6.7300000000000004,"15min":6.7300000000000004},"max":{"1min":134.21199999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":12.035},{"interface":"front","average":{"1min":14.635,"5min":14.635,"15min":14.635},"min":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"max":{"1min":27.780000000000001,"5min":27.780000000000001,"15min":27.780000000000001},"last":7.0039999999999996}]},{"osd":3,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":20.669,"5min":20.669,"15min":20.669},"min":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"max":{"1min":132.56100000000001,"5min":132.56100000000001,"15min":132.56100000000001},"last":8.0549999999999997},{"interface":"front","average":{"1min":21.187999999999999,"5min":21.187999999999999,"15min":21.187999999999999},"min":{"1min":6.5780000000000003,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":124.684,"5min":124.684,"15min":124.684},"last":14.666}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":13.888,"5min":13.888,"15min":13.888},"min":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":24.963000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":10.208},{"interface":"front","average":{"1min":15.602,"5min":15.602,"15min":15.602},"min":{"1min":8.3089999999999993,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":24.84,"5min":24.84,"15min":24.84},"last":15.241}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.725000000000001,"5min":20.725000000000001,"15min":20.725000000000001},"min":{"1min":8.3179999999999996,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":125.792,"5min":125.792,"15min":125.792},"last":12.638999999999999},{"interface":"front","average":{"1min":17.007000000000001,"5min":17.007000000000001,"15min":17.007000000000001},"min":{"1min":10.273,"5min":10.273,"15min":10.273},"max":{"1min":28.591999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":13.843999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":32.167999999999999,"5min":32.167999999999999,"15min":32.167999999999999},"min":{"1min":8.3480000000000008,"5min":8.3480000000000008,"15min":8.3480000000000008},"max":{"1min":327.29599999999999,"5min":327.29599999999999,"15min":327.29599999999999},"last":17.472000000000001},{"interface":"front","average":{"1min":33.918999999999997,"5min":33.918999999999997,"15min":33.918999999999997},"min":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"max":{"1min":323.82400000000001,"5min":323.82400000000001,"15min":323.82400000000001},"last":9.5549999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":557194,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":15.087,"5min":15.087,"15min":15.087},"min":{"1min":5.4829999999999997,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":68.513999999999996,"5min":68.513999999999996,"15min":68.513999999999996},"last":6.2729999999999997},{"interface":"front","average":{"1min":17.041,"5min":17.041,"15min":17.041},"min":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"max":{"1min":88.816000000000003,"5min":88.816000000000003,"15min":88.816000000000003},"last":9.7089999999999996}]},{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":17.635000000000002,"5min":17.635000000000002,"15min":17.635000000000002},"min":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":87.515000000000001,"5min":87.515000000000001,"15min":87.515000000000001},"last":13.541},{"interface":"front","average":{"1min":18.013000000000002,"5min":18.013000000000002,"15min":18.013000000000002},"min":{"1min":6.21,"5min":6.21,"15min":6.21},"max":{"1min":85.832999999999998,"5min":85.832999999999998,"15min":85.832999999999998},"last":12.859}]},{"osd":2,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":18.363,"5min":18.363,"15min":18.363},"min":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":113.943,"5min":113.943,"15min":113.943},"last":14.742000000000001},{"interface":"front","average":{"1min":15.470000000000001,"5min":15.470000000000001,"15min":15.470000000000001},"min":{"1min":6.5279999999999996,"5min":6.5279999999999996,"15min":6.5279999999999996},"max":{"1min":80.004000000000005,"5min":80.004000000000005,"15min":80.004000000000005},"last":7.1550000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":21.841000000000001,"5min":21.841000000000001,"15min":21.841000000000001},"min":{"1min":7.3730000000000002,"5min":7.3730000000000002,"15min":7.3730000000000002},"max":{"1min":82.506,"5min":82.506,"15min":82.506},"last":15.471},{"interface":"front","average":{"1min":23.952000000000002,"5min":23.952000000000002,"15min":23.952000000000002},"min":{"1min":9.6720000000000006,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":91.183000000000007,"5min":91.183000000000007,"15min":91.183000000000007},"last":19.355}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":12.738,"5min":12.738,"15min":12.738},"min":{"1min":7.0099999999999998,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":25.195,"5min":25.195,"15min":25.195},"last":8.8499999999999996},{"interface":"front","average":{"1min":17.186,"5min":17.186,"15min":17.186},"min":{"1min":8.7100000000000009,"5min":8.7100000000000009,"15min":8.7100000000000009},"max":{"1min":59.609000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":16.562000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":16.122,"5min":16.122,"15min":16.122},"min":{"1min":8.6820000000000004,"5min":8.6820000000000004,"15min":8.6820000000000004},"max":{"1min":66.765000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":10.593},{"interface":"front","average":{"1min":21.257999999999999,"5min":21.257999999999999,"15min":21.257999999999999},"min":{"1min":11.551,"5min":11.551,"15min":11.551},"max":{"1min":71.771000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":17.527000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.148,"5min":19.148,"15min":19.148},"min":{"1min":9.8770000000000007,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":64.677000000000007,"5min":64.677000000000007,"15min":64.677000000000007},"last":18.318999999999999},{"interface":"front","average":{"1min":18.381,"5min":18.381,"15min":18.381},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":42.531999999999996,"5min":42.531999999999996,"15min":42.531999999999996},"last":11.321999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":159354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"min":{"1min":4.5410000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":14.558,"5min":14.558,"15min":14.558},"last":8.2059999999999995},{"interface":"front","average":{"1min":10.558,"5min":10.558,"15min":10.558},"min":{"1min":6.2850000000000001,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":14.507999999999999,"5min":14.507999999999999,"15min":14.507999999999999},"last":6.9640000000000004}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":11.471,"5min":11.471,"15min":11.471},"min":{"1min":6.2960000000000003,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":19.382999999999999,"5min":19.382999999999999,"15min":19.382999999999999},"last":18.007999999999999},{"interface":"front","average":{"1min":14.153,"5min":14.153,"15min":14.153},"min":{"1min":6.609,"5min":6.609,"15min":6.609},"max":{"1min":26.864000000000001,"5min":26.864000000000001,"15min":26.864000000000001},"last":10.162000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.968,"5min":11.968,"15min":11.968},"min":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":16.492999999999999,"5min":16.492999999999999,"15min":16.492999999999999},"last":13.211},{"interface":"front","average":{"1min":15.776,"5min":15.776,"15min":15.776},"min":{"1min":12.058,"5min":12.058,"15min":12.058},"max":{"1min":21.231000000000002,"5min":21.231000000000002,"15min":21.231000000000002},"last":14.179}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.265000000000001,"5min":11.265000000000001,"15min":11.265000000000001},"min":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":29.881,"5min":29.881,"15min":29.881},"last":7.8280000000000003},{"interface":"front","average":{"1min":16.065999999999999,"5min":16.065999999999999,"15min":16.065999999999999},"min":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"max":{"1min":30.285,"5min":30.285,"15min":30.285},"last":17.065999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.603000000000002,"5min":19.603000000000002,"15min":19.603000000000002},"min":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":64.474999999999994,"5min":64.474999999999994,"15min":64.474999999999994},"last":16.135999999999999},{"interface":"front","average":{"1min":16.212,"5min":16.212,"15min":16.212},"min":{"1min":5.7770000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":75.980000000000004,"5min":75.980000000000004,"15min":75.980000000000004},"last":10.858000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.779,"5min":18.779,"15min":18.779},"min":{"1min":16.312000000000001,"5min":16.312000000000001,"15min":16.312000000000001},"max":{"1min":26.965,"5min":26.965,"15min":26.965},"last":19.471},{"interface":"front","average":{"1min":14.026,"5min":14.026,"15min":14.026},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":25.664000000000001,"5min":25.664000000000001,"15min":25.664000000000001},"last":12.074999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.358000000000001,"5min":18.358000000000001,"15min":18.358000000000001},"min":{"1min":3.5579999999999998,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":74.977999999999994,"5min":74.977999999999994,"15min":74.977999999999994},"last":15.148999999999999},{"interface":"front","average":{"1min":22.355,"5min":22.355,"15min":22.355},"min":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":83.138000000000005,"5min":83.138000000000005,"15min":83.138000000000005},"last":20.263999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:25:40.767 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:40.767 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:40.767 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:40.767 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:40.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:40.807 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:40.808 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.822 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:40.819+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:40.888 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:40.887+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:40.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:40.897 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.986 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:40.988 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:40.988 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:40.990 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:40.988+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:40.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:41.072 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:41.072 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:41.089 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:41.087+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:41.173 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:41.173 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:41.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:41.190 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:41.188+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:41.241 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:25:41.291 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:41.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:41.322 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:41.322 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:41.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:41.415 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:41.414+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:41.492 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:41.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:41.495 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:41.496 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:41.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:41.594 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:41.592+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:41.672 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:41.672 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:41.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:41.726 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:41.724+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:41.795 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:41.794+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:41.821 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:41.821 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:41.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:41.915 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:41.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:41.997 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:41.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:42.004 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.004 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:42.057 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:42.072 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:25:42.072 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T06:25:42.073 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-30T06:25:42.114 INFO:tasks.ceph.mon.c:Started 2023-10-30T06:25:42.115 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:25:42.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:42.116 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:42.112+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:42.185 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.186 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:42.199 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:42.197+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:42.301 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:42.298+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:42.328 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:42.401 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:42.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:42.466 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.466 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:42.502 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:42.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:42.603 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:42.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:42.698 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.699 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.703 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:42.702+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:42.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:42.805 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:42.803+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:42.826 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:42.849 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:42.850 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:42.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:42.945 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:42.943+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:43.006 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:43.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:43.030 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:43.030 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:43.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:43.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:43.125 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:43.124+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:43.209 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:43.208+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:43.211 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:43.211 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:43.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:43.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:25:43.310 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:43.309+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:43.397 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:43.397 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.400 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:43.400 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:43.410 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:43.409+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:43.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:43.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:43.490 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.512 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:43.510+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:43.609 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:43.609 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:43.611 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:43.612 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.612 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:43.611+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:43.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:43.699 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:43.699 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.713 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:43.711+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:43.797 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:43.798 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.813 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:43.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:43.814 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:25:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 7.5220940489999997, 2023-10-30T06:25:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:25:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:25:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:25:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:25:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:25:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:25:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:25:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:25:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:25:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:25:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:25:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:25:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:25:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:43.826 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:43.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:43.884 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:43.884 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:43.914 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:43.912+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:43.955 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:43.956 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:43.956 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:43.956 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:43.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:43.968 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:44.015 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:44.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:44.055 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:44.056 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:44.095 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:44.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:44.095 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:44.096 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:44.096 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:44.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:44.116 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:44.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:44.217 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:44.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:44.270 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:44.271 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:44.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:44.271 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:44.271 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:44.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:44.318 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:44.316+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:44.419 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:44.417+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:44.445 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:44.445 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:44.446 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:44.446 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:44.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:44.520 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:44.518+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:44.621 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:44.619+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:44.622 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:44.622 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:44.622 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:44.622 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:44.623 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:44.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:44.709 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:39.895 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.713 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:39.900 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.721 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:44.719+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:44.739 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:39.926 86065== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.762 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:46.362 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.765 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:46.365 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.788 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:46.388 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:44.822 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:44.821+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:44.849 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:44.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:44.849 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:44.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 92.691650738999996, 2023-10-30T06:25:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:44.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:44.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.852 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:44.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:44.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:44.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:44.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:44.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:44.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:44.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:44.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:44.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:44.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:44.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:44.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:44.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.863 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:44.863 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:44.863 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 89.778476875999999, 2023-10-30T06:25:44.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:44.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:44.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:44.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:44.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:44.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:44.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:44.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:44.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:44.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:44.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.874 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:44.874 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:44.874 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 89.118298906999996, 2023-10-30T06:25:44.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:44.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.875 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:44.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:44.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:44.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:44.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:44.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:44.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:44.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:44.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:44.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:44.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:44.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.886 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:44.886 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:44.886 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 89.042875699999996, 2023-10-30T06:25:44.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:44.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.887 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:44.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:44.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:44.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:44.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:44.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:44.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:44.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:44.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:44.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.897 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:44.897 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:44.897 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.771752569, 2023-10-30T06:25:44.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:44.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.898 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:44.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:44.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:44.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:44.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:44.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:44.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:44.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.904 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:44.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:44.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:44.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:44.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:44.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.908 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:44.909 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:44.909 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.741472858999998, 2023-10-30T06:25:44.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:44.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.910 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:44.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:44.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:44.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:44.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:44.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:44.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:44.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:44.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:44.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:44.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:44.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.921 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:44.922 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:44.922 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.367561774999999, 2023-10-30T06:25:44.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:44.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.923 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:44.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:44.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:44.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:44.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:44.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:44.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:44.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:44.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:44.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:44.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:44.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:44.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:44.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:44.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:44.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:44.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.933 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.934 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:44.934 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:44.934 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.363063073999996, 2023-10-30T06:25:44.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:44.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.935 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:44.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:44.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:44.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:44.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:44.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:44.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:44.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:44.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:44.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:44.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:44.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:44.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:44.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:44.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:44.945 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.945 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.945 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.946 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:44.946 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:44.946 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.071459134999998, 2023-10-30T06:25:44.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:44.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.947 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:44.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:44.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:44.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:44.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:44.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:44.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:44.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:44.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:44.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:44.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:44.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:44.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:44.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:44.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:44.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:44.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.957 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.958 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.958 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:44.958 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:44.959 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.065521817999993, 2023-10-30T06:25:44.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:44.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.959 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:44.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:44.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:44.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:44.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.966 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:44.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:44.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:44.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:44.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:44.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:44.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.971 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:44.971 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:44.971 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 88.064222803000007, 2023-10-30T06:25:44.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:44.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.972 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:44.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:44.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:44.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:44.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:44.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:44.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:44.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:44.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:44.982 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.982 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.982 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.983 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:44.983 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:44.983 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.961233995000001, 2023-10-30T06:25:44.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:44.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.984 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:44.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:44.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:44.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:44.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:44.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:44.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:44.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:44.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:44.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:44.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:44.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:44.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:44.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:44.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:44.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.994 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:44.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:44.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.995 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:44.995 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:44.995 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.689330728000002, 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:44.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:44.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:44.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:44.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:44.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:44.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:45.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:45.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:45.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:45.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:45.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:45.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:45.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:45.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:45.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:45.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:45.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:45.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.006 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:45.007 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.681832536000002, 2023-10-30T06:25:45.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:45.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.008 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:45.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:45.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:45.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:45.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:45.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:45.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:45.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:45.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:45.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:45.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:45.019 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.681171331000002, 2023-10-30T06:25:45.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:45.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.020 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:45.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:45.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:45.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:45.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:45.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:45.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:45.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:45.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:45.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:45.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:45.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:45.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:45.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.668585492999995, 2023-10-30T06:25:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.662493280000007, 2023-10-30T06:25:45.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:45.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.044 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:45.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:45.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:45.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 87.661497939, 2023-10-30T06:25:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 80.338825368000002, 2023-10-30T06:25:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 80.305383883000005, 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:45.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:45.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:45.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:45.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:45.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:45.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:45.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:45.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:45.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:45.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:45.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:45.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:45.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:45.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:45.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:45.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:45.096 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.097 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:25:45.098 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:45.099 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:45.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:45.198 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:45.197+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:45.266 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:45.267 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:45.267 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:45.267 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:45.299 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:45.297+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:45.399 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:45.398+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:45.402 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:45.402 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:45.402 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:45.403 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:45.403 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.412 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:45.500 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:45.499+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:45.538 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:45.538 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:45.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:45.538 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:45.539 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:45.600 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:45.599+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:45.677 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:45.678 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:45.678 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:45.678 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:45.727 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:45.726+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:45.754 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:45.754 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:45.754 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:25:45.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:25:45.802 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:45.801+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:45.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:45.872 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:45.872 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:45.872 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:45.872 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:45.873 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:45.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:45.903 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:45.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:46.003 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:46.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:46.066 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:46.067 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:46.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:46.067 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:46.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.068 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:25:46.068 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:46.068 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 9.0842325739999996, 2023-10-30T06:25:46.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:25:46.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:46.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:46.069 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:46.069 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:25:46.070 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:25:46.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:25:46.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.070 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:46.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:46.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:46.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:46.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:46.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:46.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:46.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:46.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:25:46.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:25:46.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:46.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:25:46.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:25:46.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:46.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:25:46.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:25:46.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:46.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:25:46.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:25:46.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:46.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:46.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:46.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:25:46.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:25:46.079 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:46.079 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:46.080 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:46.080 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:46.080 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:46.080 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:46.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:46.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:46.127 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:46.125+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:46.205 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:46.203+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:46.255 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:46.255 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:46.255 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:46.256 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:46.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:46.306 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:46.305+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:46.407 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:46.406+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:46.479 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:46.479 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.508 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:46.507+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:46.533 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:46.533 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:46.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:46.585 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:46.601 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:46.601 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:46.601 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 41, 2023-10-30T06:25:46.601 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:25:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:25:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:25:44.856160+0000", 2023-10-30T06:25:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:25:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:25:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:25:46.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5804032, 2023-10-30T06:25:46.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:25:46.603 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2023-10-30T06:25:46.603 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:25:46.603 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:25:46.604 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:25:46.604 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:25:46.604 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.604 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:25:46.604 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:25:46.605 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:25:46.606 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:25:46.606 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:25:46.606 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:25:46.606 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:25:46.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.607 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:25:46.607 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:25:46.607 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:25:46.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:25:46.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:25:46.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:25:46.608 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:25:46.608 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:25:46.608 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:25:46.608 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:25:46.609 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:25:46.610 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:25:46.610 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:25:46.610 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:25:46.610 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:25:46.610 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:25:46.611 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:25:46.611 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:25:46.611 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:25:46.611 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:25:46.611 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:25:46.612 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:25:46.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.612 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:25:46.612 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:25:46.612 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:25:46.613 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:25:46.613 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:25:46.613 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:25:46.613 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:25:46.613 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:25:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:25:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:25:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:25:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:25:46.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:25:46.615 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:25:46.615 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:25:46.615 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:25:46.615 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:25:46.615 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:25:46.616 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:25:46.617 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:25:46.617 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:25:46.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:25:46.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.617 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:25:46.618 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:25:46.619 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:25:46.619 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:25:46.619 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:25:46.619 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:25:46.619 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:25:46.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.621 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:25:46.621 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:25:46.621 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:25:46.621 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:25:46.621 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:25:46.622 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:25:46.622 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:25:46.622 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:25:46.622 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:25:46.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.623 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:25:46.623 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:25:46.623 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:25:46.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:25:46.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:25:46.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:25:46.624 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:25:46.624 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:25:46.625 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:25:46.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:25:46.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:25:46.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:25:46.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:25:46.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:25:46.627 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:25:46.627 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:25:46.627 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:25:46.627 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:25:46.627 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:25:46.628 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:25:46.629 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:25:46.629 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:25:46.629 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:25:46.629 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:25:46.629 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:25:46.630 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:25:46.630 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:25:46.630 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:25:46.630 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:25:46.630 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:25:46.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:25:46.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:25:46.631 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:25:46.631 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:25:46.631 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:25:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:25:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:25:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:25:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:25:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:25:46.633 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:25:46.633 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:25:46.633 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:25:46.633 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:25:46.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:25:46.634 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:25:46.635 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:25:46.635 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:25:46.635 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:25:46.635 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:25:46.635 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:25:46.636 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:25:46.636 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:25:46.636 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:25:46.636 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:25:46.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:25:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.630000114440918, 2023-10-30T06:25:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.630000114440918, 2023-10-30T06:25:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:25:46.637 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2023-10-30T06:25:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2023-10-30T06:25:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:25:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:25:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:25:46.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:25:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:25:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.640 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.641 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.642 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.642 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.642 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:25:46.642 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:25:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.644 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.644 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.646 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:25:46.646 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.647 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.647 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.648 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:25:46.649 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.649 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.650 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.650 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:25:46.650 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:25:46.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.652 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.652 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:25:46.652 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:25:46.652 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:25:46.653 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:25:46.653 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.653 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.653 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.654 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:25:46.654 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:25:46.654 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.655 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:25:46.656 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 33, 2023-10-30T06:25:46.656 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.656 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.656 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.656 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.657 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:25:46.657 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:25:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.660 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.660 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:25:46.660 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.662 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.662 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.663 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:25:46.664 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.664 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.664 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.664 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.665 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.665 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.665 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:25:46.665 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:25:46.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.667 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:25:46.667 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:25:46.667 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:25:46.668 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:25:46.668 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.668 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.668 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.668 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.669 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:25:46.669 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:25:46.669 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.670 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:46.671 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 26, 2023-10-30T06:25:46.671 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.671 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.671 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.671 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.672 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:25:46.672 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.673 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:25:46.673 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.674 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.674 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.676 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:25:46.676 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:25:46.677 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.678 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:25:46.678 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.679 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.679 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.679 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.679 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.681 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.681 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:25:46.681 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:25:46.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.681 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:25:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:25:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:25:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:25:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:25:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:25:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:25:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:25:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:25:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:25:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:25:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:25:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:25:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:25:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:25:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:25:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:25:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:25:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:25:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:25:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:25:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:25:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:25:46.706 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.706 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:25:46.707 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.707 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.708 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.708 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:25:46.708 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.709 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:25:46.709 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.711 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:25:46.711 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.712 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.712 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:25:46.712 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:25:46.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:25:46.713 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:25:46.714 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.714 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.714 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.714 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:25:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:25:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.715 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.716 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.716 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.716 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.716 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.716 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:46.717 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:25:46.717 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.717 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.717 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.717 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:25:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.719 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.719 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:25:46.719 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.721 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:25:46.721 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.722 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:25:46.722 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.723 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.723 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.723 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.723 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:25:46.724 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.725 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.725 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.725 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.726 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.726 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:25:46.726 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.727 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.728 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:25:46.728 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:25:46.728 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:25:46.728 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:25:46.729 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.729 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.729 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.729 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.730 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:25:46.730 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:25:46.730 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.730 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:25:46.731 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:25:46.732 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.732 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.732 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.732 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.734 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.734 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:25:46.734 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.734 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.735 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.735 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.735 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.736 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:25:46.736 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.737 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.738 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.738 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:25:46.738 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.739 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:25:46.739 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.740 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.740 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.740 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.740 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.741 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.741 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:25:46.741 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.742 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.743 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:25:46.743 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:25:46.743 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:25:46.743 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:25:46.744 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.744 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.744 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.744 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.745 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:25:46.745 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:25:46.745 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:25:46.745 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:25:46.746 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:25:46.746 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:25:46.746 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:25:46.746 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:25:46.746 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:25:46.747 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:25:46.747 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:25:46.747 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:25:46.747 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:25:46.747 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.749 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:25:46.749 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.749 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.749 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:25:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.751 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:25:46.751 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.752 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:25:46.753 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.753 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:25:46.753 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.755 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.755 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:25:46.755 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:25:46.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.756 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:25:46.756 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:25:46.756 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.757 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:25:46.757 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:25:46.757 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:25:46.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.757 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.758 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:25:46.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:25:46.758 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:25:46.758 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:25:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:25:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:25:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:25:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.760 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.761 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.761 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.761 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.761 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:25:46.761 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.762 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:25:46.762 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.762 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.763 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.763 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.763 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.763 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:25:46.763 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:25:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.764 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.765 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.765 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.765 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.765 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:25:46.765 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.766 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:25:46.766 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:25:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:25:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.768 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.769 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:25:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:25:46.770 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.771 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:25:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:25:46.772 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.773 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:25:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:25:46.774 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:25:46.775 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:25:46.775 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:25:46.775 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:25:46.775 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:25:46.776 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:25:46.776 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:25:46.776 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.776 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.776 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:25:46.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.777 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:25:46.777 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:25:46.777 INFO:teuthology.orchestra.run.smithi079.stdout: 2, 2023-10-30T06:25:46.777 INFO:teuthology.orchestra.run.smithi079.stdout: 1 2023-10-30T06:25:46.778 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.778 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.778 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.778 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:25:46.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.3", 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 7 2023-10-30T06:25:46.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.780 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:46.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.780 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:25:46.780 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:25:46.780 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:25:46.781 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:25:46.781 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:25:46.781 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:25:46.781 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:25:46.781 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:25:46.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:46.783 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:25:46.783 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:25:46.783 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:25:46.783 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:25:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:25:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:25:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:25:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:25:46.784 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:25:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:25:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:25:46.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:46.785 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:46.787 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.789 INFO:tasks.thrashosds.thrasher:j is {'epoch': 41, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:25:44.856160+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [2, 1]}], 'pg_upmap_items': [{'pgid': '2.3', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:25:46.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:25:46.790 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.791 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:46.791 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:46.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:46.792 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:46.785+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:46.810 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.888 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:46.886+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:46.895 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:46.895 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.947 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:46.947 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:46.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:46.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:46.995 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:46.995 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:46.993+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:47.084 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:47.084 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:47.089 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:47.087+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:47.092 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:47.093 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:47.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:47.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:25:47.174 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:47.175 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:47.194 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:47.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:47.286 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:47.286 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:47.290 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:47.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:47.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:47.391 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:47.390+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:47.406 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:47.407 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:25:47.422 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-30T06:25:47.273469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219720,"kb_used_data":3544,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755000,"statfs":{"total":773094113280,"available":772869120000,"internally_reserved":0,"allocated":3629056,"data_stored":2181092,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.097313"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-30T06:25:41.789033+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:25:41.789033+0000","last_peered":"2023-10-30T06:25:41.789033+0000","last_clean":"2023-10-30T06:25:41.789033+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:25:41.789033+0000","last_undegraded":"2023-10-30T06:25:41.789033+0000","last_fullsized":"2023-10-30T06:25:41.789033+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.637691+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:25:40.637691+0000","last_peered":"2023-10-30T06:25:40.637691+0000","last_clean":"2023-10-30T06:25:40.637691+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:25:40.637691+0000","last_undegraded":"2023-10-30T06:25:40.637691+0000","last_fullsized":"2023-10-30T06:25:40.637691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.190711+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:25:40.190711+0000","last_peered":"2023-10-30T06:25:40.190711+0000","last_clean":"2023-10-30T06:25:40.190711+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:25:40.190711+0000","last_undegraded":"2023-10-30T06:25:40.190711+0000","last_fullsized":"2023-10-30T06:25:40.190711+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.644832+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:25:40.644832+0000","last_peered":"2023-10-30T06:25:40.644832+0000","last_clean":"2023-10-30T06:25:40.644832+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:25:40.644832+0000","last_undegraded":"2023-10-30T06:25:40.644832+0000","last_fullsized":"2023-10-30T06:25:40.644832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-30T06:25:41.785730+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:25:41.785730+0000","last_peered":"2023-10-30T06:25:41.785730+0000","last_clean":"2023-10-30T06:25:41.785730+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:25:41.785730+0000","last_undegraded":"2023-10-30T06:25:41.785730+0000","last_fullsized":"2023-10-30T06:25:41.785730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.643319+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:25:40.643319+0000","last_peered":"2023-10-30T06:25:40.643319+0000","last_clean":"2023-10-30T06:25:40.643319+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:25:40.643319+0000","last_undegraded":"2023-10-30T06:25:40.643319+0000","last_fullsized":"2023-10-30T06:25:40.643319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-30T06:25:41.783860+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:25:41.783860+0000","last_peered":"2023-10-30T06:25:41.783860+0000","last_clean":"2023-10-30T06:25:41.783860+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:25:41.783860+0000","last_undegraded":"2023-10-30T06:25:41.783860+0000","last_fullsized":"2023-10-30T06:25:41.783860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.639818+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:25:40.639818+0000","last_peered":"2023-10-30T06:25:40.639818+0000","last_clean":"2023-10-30T06:25:40.639818+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:25:40.639818+0000","last_undegraded":"2023-10-30T06:25:40.639818+0000","last_fullsized":"2023-10-30T06:25:40.639818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":133,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-30T06:25:40.188648+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:25:40.188648+0000","last_peered":"2023-10-30T06:25:40.188648+0000","last_clean":"2023-10-30T06:25:40.188648+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:25:40.188648+0000","last_undegraded":"2023-10-30T06:25:40.188648+0000","last_fullsized":"2023-10-30T06:25:40.188648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411350,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":568252,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.359999999999999,"5min":10.359999999999999,"15min":10.359999999999999},"min":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":24.350000000000001,"5min":24.350000000000001,"15min":24.350000000000001},"last":6.5750000000000002},{"interface":"front","average":{"1min":12.083,"5min":12.083,"15min":12.083},"min":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"max":{"1min":46.798000000000002,"5min":46.798000000000002,"15min":46.798000000000002},"last":7.7240000000000002}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.798,"5min":13.798,"15min":13.798},"min":{"1min":5.5209999999999999,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":52.552,"5min":52.552,"15min":52.552},"last":12.707000000000001},{"interface":"front","average":{"1min":15.147,"5min":15.147,"15min":15.147},"min":{"1min":7.1589999999999998,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":27.946000000000002,"5min":27.946000000000002,"15min":27.946000000000002},"last":8.3290000000000006}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"min":{"1min":11.013,"5min":11.013,"15min":11.013},"max":{"1min":118.45699999999999,"5min":118.45699999999999,"15min":118.45699999999999},"last":8.9939999999999998},{"interface":"front","average":{"1min":22.571999999999999,"5min":22.571999999999999,"15min":22.571999999999999},"min":{"1min":7.8739999999999997,"5min":7.8739999999999997,"15min":7.8739999999999997},"max":{"1min":124.075,"5min":124.075,"15min":124.075},"last":11.545999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.675000000000001,"5min":13.675000000000001,"15min":13.675000000000001},"min":{"1min":7.2359999999999998,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":51.649000000000001,"5min":51.649000000000001,"15min":51.649000000000001},"last":23.103999999999999},{"interface":"front","average":{"1min":17.998999999999999,"5min":17.998999999999999,"15min":17.998999999999999},"min":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":38.271999999999998,"5min":38.271999999999998,"15min":38.271999999999998},"last":24.696000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.052,"5min":21.052,"15min":21.052},"min":{"1min":10.776,"5min":10.776,"15min":10.776},"max":{"1min":123.754,"5min":123.754,"15min":123.754},"last":15.147},{"interface":"front","average":{"1min":23.079999999999998,"5min":23.079999999999998,"15min":23.079999999999998},"min":{"1min":8.5679999999999996,"5min":8.5679999999999996,"15min":8.5679999999999996},"max":{"1min":122.72499999999999,"5min":122.72499999999999,"15min":122.72499999999999},"last":9.6400000000000006}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.457000000000001,"5min":21.457000000000001,"15min":21.457000000000001},"min":{"1min":11.9,"5min":11.9,"15min":11.9},"max":{"1min":66.400999999999996,"5min":66.400999999999996,"15min":66.400999999999996},"last":14.054},{"interface":"front","average":{"1min":19.859000000000002,"5min":19.859000000000002,"15min":19.859000000000002},"min":{"1min":8.2070000000000007,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":53.149000000000001,"5min":53.149000000000001,"15min":53.149000000000001},"last":13.326000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.513999999999999,"5min":21.513999999999999,"15min":21.513999999999999},"min":{"1min":10.657,"5min":10.657,"15min":10.657},"max":{"1min":45.235999999999997,"5min":45.235999999999997,"15min":45.235999999999997},"last":10.85},{"interface":"front","average":{"1min":21.594999999999999,"5min":21.594999999999999,"15min":21.594999999999999},"min":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":69.991,"5min":69.991,"15min":69.991},"last":15.928000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476760,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":175941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":14.706,"5min":14.706,"15min":14.706},"min":{"1min":4.8300000000000001,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":94.343000000000004,"5min":94.343000000000004,"15min":94.343000000000004},"last":8.1240000000000006},{"interface":"front","average":{"1min":15.754,"5min":15.754,"15min":15.754},"min":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":122.054,"5min":122.054,"15min":122.054},"last":49.045999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"min":{"1min":6.4379999999999997,"5min":6.4379999999999997,"15min":6.4379999999999997},"max":{"1min":18.992999999999999,"5min":18.992999999999999,"15min":18.992999999999999},"last":11.787000000000001},{"interface":"front","average":{"1min":33.863999999999997,"5min":33.863999999999997,"15min":33.863999999999997},"min":{"1min":5.9269999999999996,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":387.15600000000001,"5min":387.15600000000001,"15min":387.15600000000001},"last":10.718}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.956,"5min":19.956,"15min":19.956},"min":{"1min":6.4359999999999999,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":156.90299999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":14.484999999999999},{"interface":"front","average":{"1min":20.584,"5min":20.584,"15min":20.584},"min":{"1min":8.8940000000000001,"5min":8.8940000000000001,"15min":8.8940000000000001},"max":{"1min":143.578,"5min":143.578,"15min":143.578},"last":12.446}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.963000000000001,"5min":21.963000000000001,"15min":21.963000000000001},"min":{"1min":10.696999999999999,"5min":10.696999999999999,"15min":10.696999999999999},"max":{"1min":141.18799999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":15.44},{"interface":"front","average":{"1min":25.420999999999999,"5min":25.420999999999999,"15min":25.420999999999999},"min":{"1min":13.539999999999999,"5min":13.539999999999999,"15min":13.539999999999999},"max":{"1min":161.32900000000001,"5min":161.32900000000001,"15min":161.32900000000001},"last":17.058}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":15.045,"5min":15.045,"15min":15.045},"min":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"max":{"1min":74.079999999999998,"5min":74.079999999999998,"15min":74.079999999999998},"last":6.976},{"interface":"front","average":{"1min":15.702,"5min":15.702,"15min":15.702},"min":{"1min":9.2270000000000003,"5min":9.2270000000000003,"15min":9.2270000000000003},"max":{"1min":58.613999999999997,"5min":58.613999999999997,"15min":58.613999999999997},"last":8.6999999999999993}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":17.966000000000001,"5min":17.966000000000001,"15min":17.966000000000001},"min":{"1min":9.6560000000000006,"5min":9.6560000000000006,"15min":9.6560000000000006},"max":{"1min":71.757000000000005,"5min":71.757000000000005,"15min":71.757000000000005},"last":10.407},{"interface":"front","average":{"1min":21.762,"5min":21.762,"15min":21.762},"min":{"1min":14.734,"5min":14.734,"15min":14.734},"max":{"1min":80.119,"5min":80.119,"15min":80.119},"last":16.277999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.741,"5min":19.741,"15min":19.741},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":81.247,"5min":81.247,"15min":81.247},"last":11.429},{"interface":"front","average":{"1min":19.088000000000001,"5min":19.088000000000001,"15min":19.088000000000001},"min":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":70.989000000000004,"5min":70.989000000000004,"15min":70.989000000000004},"last":13.901}]}]},{"osd":3,"up_from":16,"seq":68719476760,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":175941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":11.06,"5min":11.06,"15min":11.06},"min":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":24.466000000000001,"5min":24.466000000000001,"15min":24.466000000000001},"last":10.234},{"interface":"front","average":{"1min":10.257999999999999,"5min":10.257999999999999,"15min":10.257999999999999},"min":{"1min":6,"5min":6,"15min":6},"max":{"1min":19.263999999999999,"5min":19.263999999999999,"15min":19.263999999999999},"last":10.709}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":26.827000000000002,"5min":26.827000000000002,"15min":26.827000000000002},"min":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":165.76400000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":22.036000000000001},{"interface":"front","average":{"1min":28.079000000000001,"5min":28.079000000000001,"15min":28.079000000000001},"min":{"1min":8.5760000000000005,"5min":8.5760000000000005,"15min":8.5760000000000005},"max":{"1min":190.34200000000001,"5min":190.34200000000001,"15min":190.34200000000001},"last":13.914}]},{"osd":2,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":18.428000000000001,"5min":18.428000000000001,"15min":18.428000000000001},"min":{"1min":12.856999999999999,"5min":12.856999999999999,"15min":12.856999999999999},"max":{"1min":41.229999999999997,"5min":41.229999999999997,"15min":41.229999999999997},"last":16.350999999999999},{"interface":"front","average":{"1min":17.199000000000002,"5min":17.199000000000002,"15min":17.199000000000002},"min":{"1min":8.1739999999999995,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":33.060000000000002,"5min":33.060000000000002,"15min":33.060000000000002},"last":23.334}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.620000000000001,"5min":20.620000000000001,"15min":20.620000000000001},"min":{"1min":9.9420000000000002,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":42.274999999999999,"5min":42.274999999999999,"15min":42.274999999999999},"last":20.552},{"interface":"front","average":{"1min":11.614000000000001,"5min":11.614000000000001,"15min":11.614000000000001},"min":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":17.463000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.658999999999999,"5min":19.658999999999999,"15min":19.658999999999999},"min":{"1min":7.4450000000000003,"5min":7.4450000000000003,"15min":7.4450000000000003},"max":{"1min":56.796999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":11.644},{"interface":"front","average":{"1min":20.798999999999999,"5min":20.798999999999999,"15min":20.798999999999999},"min":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"max":{"1min":58.576000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":28.501000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":29.138000000000002,"5min":29.138000000000002,"15min":29.138000000000002},"min":{"1min":11.694000000000001,"5min":11.694000000000001,"15min":11.694000000000001},"max":{"1min":169.05199999999999,"5min":169.05199999999999,"15min":169.05199999999999},"last":26.859999999999999},{"interface":"front","average":{"1min":15.494,"5min":15.494,"15min":15.494},"min":{"1min":9.6419999999999995,"5min":9.6419999999999995,"15min":9.6419999999999995},"max":{"1min":30.957000000000001,"5min":30.957000000000001,"15min":30.957000000000001},"last":13.009}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":17.986000000000001,"5min":17.986000000000001,"15min":17.986000000000001},"min":{"1min":8.5950000000000006,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":38.887999999999998,"5min":38.887999999999998,"15min":38.887999999999998},"last":31.125},{"interface":"front","average":{"1min":29.614999999999998,"5min":29.614999999999998,"15min":29.614999999999998},"min":{"1min":10.025,"5min":10.025,"15min":10.025},"max":{"1min":170.232,"5min":170.232,"15min":170.232},"last":29.981999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476758,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":175941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.809000000000001,"5min":27.809000000000001,"15min":27.809000000000001},"min":{"1min":5.0730000000000004,"5min":5.0730000000000004,"15min":5.0730000000000004},"max":{"1min":258.66199999999998,"5min":258.66199999999998,"15min":258.66199999999998},"last":11.959},{"interface":"front","average":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":203.38499999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":7.7210000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":16.311,"5min":16.311,"15min":16.311},"min":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":45.936999999999998,"5min":45.936999999999998,"15min":45.936999999999998},"last":16.75},{"interface":"front","average":{"1min":14.839,"5min":14.839,"15min":14.839},"min":{"1min":6.8360000000000003,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":37.390999999999998,"5min":37.390999999999998,"15min":37.390999999999998},"last":9.8170000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.326000000000001,"5min":19.326000000000001,"15min":19.326000000000001},"min":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":183.38900000000001,"5min":183.38900000000001,"15min":183.38900000000001},"last":9.5530000000000008},{"interface":"front","average":{"1min":23.067,"5min":23.067,"15min":23.067},"min":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":189.374,"5min":189.374,"15min":189.374},"last":15.647}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.959,"5min":21.959,"15min":21.959},"min":{"1min":7.7229999999999999,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":186.01400000000001,"5min":186.01400000000001,"15min":186.01400000000001},"last":12.916},{"interface":"front","average":{"1min":20.114999999999998,"5min":20.114999999999998,"15min":20.114999999999998},"min":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":167.67599999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":7.3929999999999998}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.559999999999999,"5min":27.559999999999999,"15min":27.559999999999999},"min":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":215.80500000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":17.876000000000001},{"interface":"front","average":{"1min":27.331,"5min":27.331,"15min":27.331},"min":{"1min":8.6989999999999998,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":187.863,"5min":187.863,"15min":187.863},"last":11.042999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.262,"5min":27.262,"15min":27.262},"min":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":228.81200000000001,"5min":228.81200000000001,"15min":228.81200000000001},"last":11.598000000000001},{"interface":"front","average":{"1min":30.349,"5min":30.349,"15min":30.349},"min":{"1min":9.3710000000000004,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":275.92200000000003,"5min":275.92200000000003,"15min":275.92200000000003},"last":18.984999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.227,"5min":27.227,"15min":27.227},"min":{"1min":13.772,"5min":13.772,"15min":13.772},"max":{"1min":170.43600000000001,"5min":170.43600000000001,"15min":170.43600000000001},"last":13.518000000000001},{"interface":"front","average":{"1min":23.538,"5min":23.538,"15min":23.538},"min":{"1min":8.8360000000000003,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":157.00200000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":10.459}]}]},{"osd":0,"up_from":16,"seq":68719476758,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":175941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":10.695,"5min":10.695,"15min":10.695},"min":{"1min":6.109,"5min":6.109,"15min":6.109},"max":{"1min":25.989999999999998,"5min":25.989999999999998,"15min":25.989999999999998},"last":39.700000000000003},{"interface":"front","average":{"1min":11.907999999999999,"5min":11.907999999999999,"15min":11.907999999999999},"min":{"1min":5.0010000000000003,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":30.788,"5min":30.788,"15min":30.788},"last":28.358000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.052,"5min":20.052,"15min":20.052},"min":{"1min":6.5110000000000001,"5min":6.5110000000000001,"15min":6.5110000000000001},"max":{"1min":127.95399999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":9.4689999999999994},{"interface":"front","average":{"1min":14.872,"5min":14.872,"15min":14.872},"min":{"1min":7.5720000000000001,"5min":7.5720000000000001,"15min":7.5720000000000001},"max":{"1min":35.834000000000003,"5min":35.834000000000003,"15min":35.834000000000003},"last":11.839}]},{"osd":3,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.649999999999999,"5min":22.649999999999999,"15min":22.649999999999999},"min":{"1min":8.984,"5min":8.984,"15min":8.984},"max":{"1min":99.790999999999997,"5min":99.790999999999997,"15min":99.790999999999997},"last":12.646000000000001},{"interface":"front","average":{"1min":16.715,"5min":16.715,"15min":16.715},"min":{"1min":9.4689999999999994,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":26.454999999999998,"5min":26.454999999999998,"15min":26.454999999999998},"last":15.948}]},{"osd":4,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":16.649999999999999,"5min":16.649999999999999,"15min":16.649999999999999},"min":{"1min":10.246,"5min":10.246,"15min":10.246},"max":{"1min":41.134999999999998,"5min":41.134999999999998,"15min":41.134999999999998},"last":10.5},{"interface":"front","average":{"1min":25.373999999999999,"5min":25.373999999999999,"15min":25.373999999999999},"min":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"max":{"1min":103.705,"5min":103.705,"15min":103.705},"last":17.295999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":19.039000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"min":{"1min":12.388999999999999,"5min":12.388999999999999,"15min":12.388999999999999},"max":{"1min":43.847000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":15.16},{"interface":"front","average":{"1min":28.347999999999999,"5min":28.347999999999999,"15min":28.347999999999999},"min":{"1min":12.076000000000001,"5min":12.076000000000001,"15min":12.076000000000001},"max":{"1min":98.707999999999998,"5min":98.707999999999998,"15min":98.707999999999998},"last":20.706}]},{"osd":6,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.504000000000001,"5min":20.504000000000001,"15min":20.504000000000001},"min":{"1min":8.6869999999999994,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":43.192999999999998,"5min":43.192999999999998,"15min":43.192999999999998},"last":18.431999999999999},{"interface":"front","average":{"1min":22.300999999999998,"5min":22.300999999999998,"15min":22.300999999999998},"min":{"1min":13.728,"5min":13.728,"15min":13.728},"max":{"1min":44.545000000000002,"5min":44.545000000000002,"15min":44.545000000000002},"last":11.210000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.123999999999999,"5min":22.123999999999999,"15min":22.123999999999999},"min":{"1min":11.917999999999999,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":45.927999999999997,"5min":45.927999999999997,"15min":45.927999999999997},"last":19.766999999999999},{"interface":"front","average":{"1min":23.283000000000001,"5min":23.283000000000001,"15min":23.283000000000001},"min":{"1min":13.298,"5min":13.298,"15min":13.298},"max":{"1min":45.130000000000003,"5min":45.130000000000003,"15min":45.130000000000003},"last":22.199999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":170412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":16.344999999999999,"5min":16.344999999999999,"15min":16.344999999999999},"min":{"1min":4.4249999999999998,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":131.477,"5min":131.477,"15min":131.477},"last":7.9569999999999999},{"interface":"front","average":{"1min":16.484000000000002,"5min":16.484000000000002,"15min":16.484000000000002},"min":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":129.386,"5min":129.386,"15min":129.386},"last":12.247}]},{"osd":1,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":10.994,"5min":10.994,"15min":10.994},"min":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"max":{"1min":25.774000000000001,"5min":25.774000000000001,"15min":25.774000000000001},"last":7.1870000000000003},{"interface":"front","average":{"1min":19.489000000000001,"5min":19.489000000000001,"15min":19.489000000000001},"min":{"1min":9.9960000000000004,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":91.295000000000002,"5min":91.295000000000002,"15min":91.295000000000002},"last":14.018000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":18.219000000000001,"5min":18.219000000000001,"15min":18.219000000000001},"min":{"1min":6.7300000000000004,"5min":6.7300000000000004,"15min":6.7300000000000004},"max":{"1min":134.21199999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":10.657999999999999},{"interface":"front","average":{"1min":14.635,"5min":14.635,"15min":14.635},"min":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"max":{"1min":27.780000000000001,"5min":27.780000000000001,"15min":27.780000000000001},"last":8.6080000000000005}]},{"osd":3,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":20.669,"5min":20.669,"15min":20.669},"min":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"max":{"1min":132.56100000000001,"5min":132.56100000000001,"15min":132.56100000000001},"last":16.058},{"interface":"front","average":{"1min":21.187999999999999,"5min":21.187999999999999,"15min":21.187999999999999},"min":{"1min":6.5780000000000003,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":124.684,"5min":124.684,"15min":124.684},"last":14.653}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":13.888,"5min":13.888,"15min":13.888},"min":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":24.963000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":9.5039999999999996},{"interface":"front","average":{"1min":15.602,"5min":15.602,"15min":15.602},"min":{"1min":8.3089999999999993,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":24.84,"5min":24.84,"15min":24.84},"last":17.314}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.725000000000001,"5min":20.725000000000001,"15min":20.725000000000001},"min":{"1min":8.3179999999999996,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":125.792,"5min":125.792,"15min":125.792},"last":10.074},{"interface":"front","average":{"1min":17.007000000000001,"5min":17.007000000000001,"15min":17.007000000000001},"min":{"1min":10.273,"5min":10.273,"15min":10.273},"max":{"1min":28.591999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":18.085000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":32.167999999999999,"5min":32.167999999999999,"15min":32.167999999999999},"min":{"1min":8.3480000000000008,"5min":8.3480000000000008,"15min":8.3480000000000008},"max":{"1min":327.29599999999999,"5min":327.29599999999999,"15min":327.29599999999999},"last":15.353},{"interface":"front","average":{"1min":33.918999999999997,"5min":33.918999999999997,"15min":33.918999999999997},"min":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"max":{"1min":323.82400000000001,"5min":323.82400000000001,"15min":323.82400000000001},"last":11.369999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":568252,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":15.087,"5min":15.087,"15min":15.087},"min":{"1min":5.4829999999999997,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":68.513999999999996,"5min":68.513999999999996,"15min":68.513999999999996},"last":10.237},{"interface":"front","average":{"1min":17.041,"5min":17.041,"15min":17.041},"min":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"max":{"1min":88.816000000000003,"5min":88.816000000000003,"15min":88.816000000000003},"last":10.808999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":17.635000000000002,"5min":17.635000000000002,"15min":17.635000000000002},"min":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":87.515000000000001,"5min":87.515000000000001,"15min":87.515000000000001},"last":7.3710000000000004},{"interface":"front","average":{"1min":18.013000000000002,"5min":18.013000000000002,"15min":18.013000000000002},"min":{"1min":6.21,"5min":6.21,"15min":6.21},"max":{"1min":85.832999999999998,"5min":85.832999999999998,"15min":85.832999999999998},"last":13.882}]},{"osd":2,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":18.363,"5min":18.363,"15min":18.363},"min":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":113.943,"5min":113.943,"15min":113.943},"last":16.356000000000002},{"interface":"front","average":{"1min":15.470000000000001,"5min":15.470000000000001,"15min":15.470000000000001},"min":{"1min":6.5279999999999996,"5min":6.5279999999999996,"15min":6.5279999999999996},"max":{"1min":80.004000000000005,"5min":80.004000000000005,"15min":80.004000000000005},"last":6.5369999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":21.841000000000001,"5min":21.841000000000001,"15min":21.841000000000001},"min":{"1min":7.3730000000000002,"5min":7.3730000000000002,"15min":7.3730000000000002},"max":{"1min":82.506,"5min":82.506,"15min":82.506},"last":13.039999999999999},{"interface":"front","average":{"1min":23.952000000000002,"5min":23.952000000000002,"15min":23.952000000000002},"min":{"1min":9.6720000000000006,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":91.183000000000007,"5min":91.183000000000007,"15min":91.183000000000007},"last":17.181000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":12.738,"5min":12.738,"15min":12.738},"min":{"1min":7.0099999999999998,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":25.195,"5min":25.195,"15min":25.195},"last":11.461},{"interface":"front","average":{"1min":17.186,"5min":17.186,"15min":17.186},"min":{"1min":8.7100000000000009,"5min":8.7100000000000009,"15min":8.7100000000000009},"max":{"1min":59.609000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":14.516}]},{"osd":6,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":16.122,"5min":16.122,"15min":16.122},"min":{"1min":8.6820000000000004,"5min":8.6820000000000004,"15min":8.6820000000000004},"max":{"1min":66.765000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":8.2080000000000002},{"interface":"front","average":{"1min":21.257999999999999,"5min":21.257999999999999,"15min":21.257999999999999},"min":{"1min":11.551,"5min":11.551,"15min":11.551},"max":{"1min":71.771000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":15.106999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.148,"5min":19.148,"15min":19.148},"min":{"1min":9.8770000000000007,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":64.677000000000007,"5min":64.677000000000007,"15min":64.677000000000007},"last":18.079999999999998},{"interface":"front","average":{"1min":18.381,"5min":18.381,"15min":18.381},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":42.531999999999996,"5min":42.531999999999996,"15min":42.531999999999996},"last":9.5809999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":170412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"min":{"1min":4.5410000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":14.558,"5min":14.558,"15min":14.558},"last":5.6379999999999999},{"interface":"front","average":{"1min":10.558,"5min":10.558,"15min":10.558},"min":{"1min":6.2850000000000001,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":14.507999999999999,"5min":14.507999999999999,"15min":14.507999999999999},"last":8.1519999999999992}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":11.471,"5min":11.471,"15min":11.471},"min":{"1min":6.2960000000000003,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":19.382999999999999,"5min":19.382999999999999,"15min":19.382999999999999},"last":13.153},{"interface":"front","average":{"1min":14.153,"5min":14.153,"15min":14.153},"min":{"1min":6.609,"5min":6.609,"15min":6.609},"max":{"1min":26.864000000000001,"5min":26.864000000000001,"15min":26.864000000000001},"last":10.183999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.968,"5min":11.968,"15min":11.968},"min":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":16.492999999999999,"5min":16.492999999999999,"15min":16.492999999999999},"last":12.151999999999999},{"interface":"front","average":{"1min":15.776,"5min":15.776,"15min":15.776},"min":{"1min":12.058,"5min":12.058,"15min":12.058},"max":{"1min":21.231000000000002,"5min":21.231000000000002,"15min":21.231000000000002},"last":9.625}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.265000000000001,"5min":11.265000000000001,"15min":11.265000000000001},"min":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":29.881,"5min":29.881,"15min":29.881},"last":11.583},{"interface":"front","average":{"1min":16.065999999999999,"5min":16.065999999999999,"15min":16.065999999999999},"min":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"max":{"1min":30.285,"5min":30.285,"15min":30.285},"last":14.722}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.603000000000002,"5min":19.603000000000002,"15min":19.603000000000002},"min":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":64.474999999999994,"5min":64.474999999999994,"15min":64.474999999999994},"last":13.771000000000001},{"interface":"front","average":{"1min":16.212,"5min":16.212,"15min":16.212},"min":{"1min":5.7770000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":75.980000000000004,"5min":75.980000000000004,"15min":75.980000000000004},"last":8.6189999999999998}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.779,"5min":18.779,"15min":18.779},"min":{"1min":16.312000000000001,"5min":16.312000000000001,"15min":16.312000000000001},"max":{"1min":26.965,"5min":26.965,"15min":26.965},"last":15.996},{"interface":"front","average":{"1min":14.026,"5min":14.026,"15min":14.026},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":25.664000000000001,"5min":25.664000000000001,"15min":25.664000000000001},"last":18.077999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.358000000000001,"5min":18.358000000000001,"15min":18.358000000000001},"min":{"1min":3.5579999999999998,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":74.977999999999994,"5min":74.977999999999994,"15min":74.977999999999994},"last":15.319000000000001},{"interface":"front","average":{"1min":22.355,"5min":22.355,"15min":22.355},"min":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":83.138000000000005,"5min":83.138000000000005,"15min":83.138000000000005},"last":17.251000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:25:47.425 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [5, 0, 1, 7] 2023-10-30T06:25:47.425 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.7', '5', '0', '1', '7'] 2023-10-30T06:25:47.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.7 5 0 1 7 2023-10-30T06:25:47.466 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:47.466 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:47.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:47.518 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:47.515+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:47.593 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:47.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:47.664 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:47.664 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:47.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:47.716 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:47.713+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:47.795 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:47.793+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:47.856 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:47.856 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:47.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:47.906 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:47.905+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:47.997 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:47.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:48.034 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.034 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:48.128 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:48.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:48.199 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:48.197+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:48.211 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.212 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:48.300 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:48.299+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:48.342 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.7 pg_upmap_items mapping to [5->0,1->7] 2023-10-30T06:25:48.366 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:25:48.395 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.395 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.400 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:48.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:48.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:48.501 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:48.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:48.539 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.540 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:48.632 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:48.631+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:48.703 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:48.702+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:48.713 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.713 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:48.813 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:48.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:48.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:48.905 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:48.904+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:49.006 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:49.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:49.030 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.030 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:49.107 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:49.106+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:49.164 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.164 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:49.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:49.216 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:49.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:49.302 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.303 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.308 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:49.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:49.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:49.409 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:49.408+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:49.431 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:49.432 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.481 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.481 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:49.519 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:49.519 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.531 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:49.530+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:49.609 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:49.609 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.613 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:49.611+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:49.657 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.657 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:49.697 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:49.698 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.711 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:49.710+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:49.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:49.795 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.806 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.806 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.813 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:49.811+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:49.815 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:49.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:49.882 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.913 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:49.912+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:49.983 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:49.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:49.990 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:49.990 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:49.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:50.043 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:50.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:50.072 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:50.073 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:50.115 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:50.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:50.127 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:50.127 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:50.127 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:50.128 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:50.128 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:50.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:50.216 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:50.214+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:50.304 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:50.304 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:50.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:50.305 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:50.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.305 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:25:50.305 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 13.331474288000001, 2023-10-30T06:25:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:25:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:50.306 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:50.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:50.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:50.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:50.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:50.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:25:50.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:50.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:50.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:25:50.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:25:50.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:50.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:25:50.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:25:50.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:50.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:25:50.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:25:50.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:50.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:25:50.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:25:50.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:50.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:25:50.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:25:50.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:50.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:50.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:50.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:25:50.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:25:50.316 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:50.316 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:50.316 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:50.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:50.317 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:50.317 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:50.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:50.319 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:50.317+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:50.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:25:50.463 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:50.461+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:50.474 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:50.475 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:50.475 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:50.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:50.536 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:50.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:50.580 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.405 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:50.621 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:50.619+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:50.667 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:50.668 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:50.668 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:50.668 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:50.668 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:50.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:50.722 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:50.721+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:50.823 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:50.821+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:50.831 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.655 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:50.855 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:50.855 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:50.856 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:50.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:50.924 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:50.923+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:51.024 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:51.023+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:51.034 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:51.034 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:51.035 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:51.035 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:51.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:51.128 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:51.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:51.211 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:51.211 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:51.212 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:51.212 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:51.212 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:51.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:51.262 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:51.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:51.327 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:51.326+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:51.428 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:51.427+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:51.447 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.106535120999993, 2023-10-30T06:25:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.785379930000005, 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.781135372999998, 2023-10-30T06:25:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.695676947999999, 2023-10-30T06:25:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.426433732000007, 2023-10-30T06:25:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.425602321, 2023-10-30T06:25:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.424935194, 2023-10-30T06:25:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.402661123000001, 2023-10-30T06:25:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.402013267000001, 2023-10-30T06:25:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 95.401327194000004, 2023-10-30T06:25:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 94.330445225999995, 2023-10-30T06:25:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 94.329781260000004, 2023-10-30T06:25:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 94.301327451000006, 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.959560957999997, 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.958812288000004, 2023-10-30T06:25:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.958143905, 2023-10-30T06:25:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.948293230999994, 2023-10-30T06:25:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.947619415999995, 2023-10-30T06:25:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.946971032999997, 2023-10-30T06:25:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 86.942854566999998, 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:51.768 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:51.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:51.770 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:51.769+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:51.870 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:51.869+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:51.873 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:25:51.948 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:51.949 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:51.949 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:51.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:51.971 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:51.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:52.073 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:52.071+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:52.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:52.087 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:52.087 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:52.088 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:52.088 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:52.088 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:52.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:52.183 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:52.181+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:52.275 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:52.274+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:52.282 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:52.282 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:52.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:52.283 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:52.283 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:52.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:52.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:52.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.376 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:52.374+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:52.447 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:52.448 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.477 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:52.475+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:52.478 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:52.478 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:52.478 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:52.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:52.548 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:52.552 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.577 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:52.576+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:52.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:52.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.673 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:52.673 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:52.678 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:52.676+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:52.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:52.730 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:52.731 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.779 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:52.777+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:52.817 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:52.818 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.851 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:52.852 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:52.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:52.903 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:52.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:52.919 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:52.919 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:52.980 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:52.979+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:53.023 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.023 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.033 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:53.037 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:53.037 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:53.090 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:53.088+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:53.163 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.163 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:53.217 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:53.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:53.283 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:53.281+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:53.300 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.300 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:53.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:53.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:53.394 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:53.393+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:53.476 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.476 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:53.485 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:53.483+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:53.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:53.586 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:53.584+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:53.654 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.654 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:53.704 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:53.703+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:53.788 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:53.786+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:53.831 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:53.831 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:53.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:53.889 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:53.887+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:53.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:25:53.998 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:53.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:54.013 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.013 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:54.091 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:54.089+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:54.131 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:54.189 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:12.013 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:54.191 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.191 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.192 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:54.190+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:54.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:54.243 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:12.067 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:54.292 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:54.291+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:54.327 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.327 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:54.393 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:54.392+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:54.462 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.463 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:54.493 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:12.317 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:25:54.494 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:54.493+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:54.596 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:25:54.594+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:25:54.597 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.597 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:54.697 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:54.695+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:54.736 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.736 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:54.798 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:54.797+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:54.891 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:54.892 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:54.898 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:54.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:54.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:55.000 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:54.998+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:55.029 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:55.029 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:55.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:55.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:25:55.123 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:55.121+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:55.202 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:55.200+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:55.217 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:55.218 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:55.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:55.302 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:55.301+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:55.373 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:55.374 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:25:55.403 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:55.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:55.412 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:55.413 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:55.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:25:55.504 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:55.503+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:55.507 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:55.508 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:25:55.556 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:55.557 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:55.557 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:55.557 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:55.557 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:55.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:25:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:55.608 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:25:55.608 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:55.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:55.694 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:25:55.705 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:55.704+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:55.757 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:55.757 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:55.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:55.757 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:55.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.758 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:25:55.758 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:55.758 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 102.417779614, 2023-10-30T06:25:55.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:25:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.759 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:25:55.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.760 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:55.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:25:55.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:25:55.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:25:55.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:25:55.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:25:55.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:25:55.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:25:55.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:25:55.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:25:55.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:25:55.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:25:55.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:25:55.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:25:55.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:25:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:25:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:25:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:25:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:25:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:25:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.09662442299999, 2023-10-30T06:25:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:25:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:25:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:25:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:25:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:25:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:25:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:25:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:25:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:25:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:25:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:25:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:25:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:25:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:25:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:25:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:25:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:25:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:25:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:25:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:25:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:25:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.092379866, 2023-10-30T06:25:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:25:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:25:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:25:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:25:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:25:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:25:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:25:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:25:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:25:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:25:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:25:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:25:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:25:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:25:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:25:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:25:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:25:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:25:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:25:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:25:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:25:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:25:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:25:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.006921441, 2023-10-30T06:25:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:25:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:25:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:25:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:25:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:25:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:25:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:25:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:25:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:25:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:25:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:25:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:25:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:25:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:25:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:25:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:25:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:25:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:25:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:25:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:25:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:25:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:25:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:25:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:25:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:25:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.737678224999996, 2023-10-30T06:25:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:25:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:25:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:25:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:25:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:25:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:25:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:25:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:25:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:25:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:25:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:25:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:25:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:25:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:25:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:25:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:25:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:25:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:25:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:25:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:25:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:25:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.736846814000003, 2023-10-30T06:25:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:25:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:25:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:25:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:25:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:25:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:25:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:25:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:25:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:25:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:25:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:25:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:25:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:25:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:25:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:25:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:25:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:25:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:25:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:25:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:25:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:25:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:25:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:25:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.736179687000003, 2023-10-30T06:25:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:25:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:25:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:25:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:25:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:25:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:25:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:25:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:25:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:25:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:25:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:25:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:25:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:25:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:25:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:25:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:25:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:25:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:25:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:25:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:25:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:25:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.713905616000005, 2023-10-30T06:25:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:25:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:25:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:25:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:25:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:25:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:25:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:25:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:25:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:25:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:25:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:25:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:25:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:25:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:25:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:25:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:25:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:25:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:25:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:25:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:25:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:25:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:25:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:25:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:25:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:25:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.713257760000005, 2023-10-30T06:25:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:25:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:25:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:25:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:25:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:25:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:25:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:25:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:25:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:25:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:25:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:25:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:25:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:25:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:25:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:25:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:25:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:25:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:25:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:25:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:25:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:25:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:25:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:25:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:25:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:25:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:25:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 99.712571686999993, 2023-10-30T06:25:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:25:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:25:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:25:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:25:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:25:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:25:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:25:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:25:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:25:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:25:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:25:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:25:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:25:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:25:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:25:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:25:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:25:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:25:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:25:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:25:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:25:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:25:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:25:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:25:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:25:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.641689718999999, 2023-10-30T06:25:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:25:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:25:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:25:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:25:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:25:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:25:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:25:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:25:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:25:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:25:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:25:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:25:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:25:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:25:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:25:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:25:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:25:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:25:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:25:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:25:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:25:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:25:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:25:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:25:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:25:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.641025752999994, 2023-10-30T06:25:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:25:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:25:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:25:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:25:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:25:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:25:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:25:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:25:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:25:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:25:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:25:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:25:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:25:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:25:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:25:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:25:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:25:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:25:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:25:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:25:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:25:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:25:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:25:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 98.612571943999995, 2023-10-30T06:25:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:25:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:25:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:25:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:25:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:25:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:25:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:25:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:25:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:25:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:25:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:25:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:25:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:25:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:25:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:25:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:25:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:25:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:25:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:25:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:25:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.270805451000001, 2023-10-30T06:25:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:25:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:25:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:25:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:25:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:25:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:25:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:25:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:25:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:25:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:25:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:25:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:25:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:25:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:25:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:25:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:25:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:25:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:25:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:25:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:25:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:25:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:25:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:25:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.270056780999994, 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:25:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:25:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:25:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:25:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:25:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:25:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:25:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:25:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:25:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:25:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:25:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:25:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:25:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:25:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:25:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:25:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:25:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:25:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:25:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:25:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:25:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:25:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.269388398000004, 2023-10-30T06:25:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:25:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:25:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:25:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:25:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:25:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:25:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:25:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:25:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:25:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:25:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:25:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:25:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:25:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:25:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:25:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:25:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:25:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:25:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:25:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:25:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:25:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:25:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:25:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.259537723999998, 2023-10-30T06:25:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:25:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:25:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:25:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:25:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:25:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:25:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:25:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:25:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:25:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:25:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:25:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:25:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:25:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:25:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:25:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:25:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:25:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:25:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:25:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:25:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:25:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:25:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:25:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:25:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.258863908999999, 2023-10-30T06:25:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:25:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:25:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:25:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:25:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:25:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:25:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:25:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:25:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:25:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:25:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:25:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:25:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:25:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:25:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:25:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:25:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:25:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:25:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:25:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:25:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:25:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:25:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:25:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.258215526000001, 2023-10-30T06:25:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:25:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:25:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:25:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:25:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:25:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:25:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:25:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:25:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:25:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:25:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:25:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:25:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:25:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:25:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:25:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:25:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:25:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:25:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:25:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:25:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:25:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:25:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:25:56.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 91.254099060000001, 2023-10-30T06:25:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:25:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:56.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:25:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:25:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.089 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:25:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:25:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:25:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:25:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:25:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:25:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:25:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:25:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:25:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:25:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:25:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:25:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:25:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:25:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:25:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:25:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:25:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:25:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:25:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:25:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:25:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:56.103 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:25:56.105 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:56.105 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:25:56.106 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:25:56.107 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:56.107 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:25:56.108 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:56.108 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:25:56.108 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:56.109 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:56.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:56.110 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:25:56.205 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:56.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:56.275 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:56.275 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:25:56.329 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:56.327+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:56.406 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:56.405+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:56.412 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:56.413 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:56.413 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:56.413 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:56.413 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:25:56.507 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:56.506+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:56.595 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:56.595 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:56.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:56.597 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:25:56.597 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:25:56.647 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:56.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:56.708 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:56.707+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:56.734 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:56.734 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:25:56.735 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:25:56.809 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:56.808+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:56.909 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:56.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:56.910 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:25:56.910 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:25:56.911 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:25:56.911 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:56.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:25:57.010 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:57.008+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:57.089 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:25:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:25:57.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:25:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:25:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:25:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:57.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 20.797368858999999, 2023-10-30T06:25:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:25:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:25:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:25:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:25:57.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:25:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:25:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:25:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:25:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:25:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:25:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:25:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:25:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:25:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:25:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:25:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:25:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:25:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:25:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:25:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:25:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:25:57.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:25:57.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:25:57.101 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:25:57.102 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:25:57.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:25:57.146 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:25:57.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:25:57.212 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:57.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:57.228 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:57.229 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:57.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:25:57.321 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:25:57.320+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:25:57.403 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:57.404 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:57.404 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:57.404 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:57.404 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:57.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:25:57.414 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:57.412+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:57.514 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:57.513+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:57.583 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:57.584 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:57.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:57.584 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:57.584 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:57.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:25:57.615 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:57.613+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:57.716 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:57.715+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:57.721 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:57.721 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:57.722 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:57.722 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:57.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:25:57.762 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:25:57.774 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:25:57.774 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T06:25:57.774 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:25:57.811 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:25:57.794+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 86060) UID: 0 2023-10-30T06:25:57.812 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:25:57.796+0000 9f64640 -1 mon.b@1(leader) e1 *** Got Signal Terminated *** 2023-10-30T06:25:57.862 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:57.860+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:57.917 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:57.916+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:57.959 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:57.959 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:57.960 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:57.960 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:57.960 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:57.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:25:58.056 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:58.054+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:58.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:25:58.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:25:58.119 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:58.118+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:58.184 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:58.184 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:58.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:58.184 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:25:58.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.185 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:25:58.185 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:58.185 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.02450509000001, 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:25:58.186 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:25:58.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:25:58.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.187 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:58.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:25:58.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:25:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:25:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:25:58.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:25:58.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:25:58.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:25:58.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:25:58.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:25:58.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:25:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:25:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:25:58.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:25:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:25:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:25:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:25:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:25:58.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:25:58.199 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.199 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.199 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 103.11133122699999, 2023-10-30T06:25:58.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:25:58.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.201 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:58.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:25:58.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:25:58.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:25:58.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:25:58.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:25:58.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:25:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:25:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:25:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:25:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:25:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:25:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:25:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:25:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:25:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:25:58.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:25:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:25:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:25:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.45115325800001, 2023-10-30T06:25:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:25:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:25:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:25:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:25:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:25:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:25:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:25:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:25:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:25:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:25:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:25:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:25:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:25:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:25:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:25:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.37573005100001, 2023-10-30T06:25:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:25:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:25:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:25:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:25:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:25:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:25:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:25:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:25:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:25:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:25:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:25:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:25:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:25:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.10460691999999, 2023-10-30T06:25:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:25:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:25:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:25:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:25:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:25:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:25:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:25:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:25:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:25:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:25:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:25:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:25:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:25:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.07432721000001, 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:25:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:25:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:25:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:25:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:25:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:25:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:25:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:25:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:25:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:25:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:25:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.70041612599999, 2023-10-30T06:25:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:25:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:25:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:25:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:25:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:25:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:25:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:25:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:25:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:25:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:25:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:25:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:25:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:25:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:25:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:25:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.695917425, 2023-10-30T06:25:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:25:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:25:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:25:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:25:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:25:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:25:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:25:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:25:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:25:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:25:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:25:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:25:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:25:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.40431348600001, 2023-10-30T06:25:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:25:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:25:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:25:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:25:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:25:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:25:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:25:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:25:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:25:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:25:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:25:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:25:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.398376169, 2023-10-30T06:25:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:25:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:25:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:25:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:25:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:25:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:25:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:25:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:25:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:25:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:25:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:25:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:25:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:25:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.397077154, 2023-10-30T06:25:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:25:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:25:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:25:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:25:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:25:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:25:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:25:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:25:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:25:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:25:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:25:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:25:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:25:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.294088346, 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:25:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:25:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:25:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:25:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:25:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:25:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:25:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:25:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:25:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:25:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:25:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.022185079, 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:25:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:25:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:25:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:25:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:25:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:25:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:25:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:25:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:25:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:25:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:25:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.014686887, 2023-10-30T06:25:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:25:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:25:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:25:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:25:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:25:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:25:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:25:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:25:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:25:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:25:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:25:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:25:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.014025682, 2023-10-30T06:25:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:25:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:25:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:25:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:25:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:25:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:25:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:25:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:25:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:25:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:25:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:25:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:25:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 101.001439844, 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:25:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:25:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:25:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:25:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:25:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:25:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:25:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:25:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:25:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:25:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:25:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:25:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 100.995347631, 2023-10-30T06:25:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:25:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:25:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:25:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:25:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:25:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:25:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:25:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:25:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:25:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:25:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:25:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:25:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:25:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 100.99435228999999, 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:25:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:25:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:25:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:25:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:25:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:25:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:25:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:25:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:25:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 93.671679718999997, 2023-10-30T06:25:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:25:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:25:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:25:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:25:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:25:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:25:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:25:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:25:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:25:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:25:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:25:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:25:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:25:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:25:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:25:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:25:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 93.638238233999999, 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:25:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:25:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:25:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:25:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:25:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:25:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:25:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:25:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:25:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:25:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:25:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:25:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:25:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:25:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:25:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:25:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:25:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:25:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:25:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:25:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:25:58.429 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:58.430 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:25:58.430 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:25:58.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:25:58.434 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:25:58.434 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:25:58.436 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:58.430+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:58.504 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:25:58.505 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.547 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:58.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:58.602 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:58.602 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:58.603 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:25:58.603 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:58.609 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:25:58.610 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:25:58.657 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:58.655+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:58.708 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:25:58.708 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.735 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:58.733+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:58.743 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:58.743 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:25:58.743 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:25:58.744 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:25:58.744 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:58.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:25:58.811 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:25:58.811 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.835 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:58.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:58.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:25:58.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:25:58.936 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:25:58.936 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:25:58.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:25:58.936 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:25:58.937 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:25:58.937 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:58.935+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:58.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:25:58.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:25:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:25:59.038 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:59.036+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:59.046 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:25:59.049 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:25:59.059 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-10-30T06:25:57.275066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219968,"kb_used_data":3808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754752,"statfs":{"total":773094113280,"available":772868866048,"internally_reserved":0,"allocated":3899392,"data_stored":2302866,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":88,"apply_latency_ms":88,"commit_latency_ns":88000000,"apply_latency_ns":88000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001216"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.132621+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:25:52.132621+0000","last_peered":"2023-10-30T06:25:52.132621+0000","last_clean":"2023-10-30T06:25:52.132621+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:25:52.132621+0000","last_undegraded":"2023-10-30T06:25:52.132621+0000","last_fullsized":"2023-10-30T06:25:52.132621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.503714+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:25:48.503714+0000","last_peered":"2023-10-30T06:25:48.503714+0000","last_clean":"2023-10-30T06:25:48.503714+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:25:48.503714+0000","last_undegraded":"2023-10-30T06:25:48.503714+0000","last_fullsized":"2023-10-30T06:25:48.503714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.812123+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:25:48.812123+0000","last_peered":"2023-10-30T06:25:48.812123+0000","last_clean":"2023-10-30T06:25:48.812123+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:25:48.812123+0000","last_undegraded":"2023-10-30T06:25:48.812123+0000","last_fullsized":"2023-10-30T06:25:48.812123+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.372209+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:25:48.372209+0000","last_peered":"2023-10-30T06:25:48.372209+0000","last_clean":"2023-10-30T06:25:48.372209+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:25:48.372209+0000","last_undegraded":"2023-10-30T06:25:48.372209+0000","last_fullsized":"2023-10-30T06:25:48.372209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.130157+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:25:52.130157+0000","last_peered":"2023-10-30T06:25:52.130157+0000","last_clean":"2023-10-30T06:25:52.130157+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:25:52.130157+0000","last_undegraded":"2023-10-30T06:25:52.130157+0000","last_fullsized":"2023-10-30T06:25:52.130157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.818212+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:25:48.818212+0000","last_peered":"2023-10-30T06:25:48.818212+0000","last_clean":"2023-10-30T06:25:48.818212+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:25:48.818212+0000","last_undegraded":"2023-10-30T06:25:48.818212+0000","last_fullsized":"2023-10-30T06:25:48.818212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.126002+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:25:52.126002+0000","last_peered":"2023-10-30T06:25:52.126002+0000","last_clean":"2023-10-30T06:25:52.126002+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:25:52.126002+0000","last_undegraded":"2023-10-30T06:25:52.126002+0000","last_fullsized":"2023-10-30T06:25:52.126002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.820662+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:25:48.820662+0000","last_peered":"2023-10-30T06:25:48.820662+0000","last_clean":"2023-10-30T06:25:48.820662+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:25:48.820662+0000","last_undegraded":"2023-10-30T06:25:48.820662+0000","last_fullsized":"2023-10-30T06:25:48.820662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":136,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-30T06:25:48.809820+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:25:48.809820+0000","last_peered":"2023-10-30T06:25:48.809820+0000","last_clean":"2023-10-30T06:25:48.809820+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:25:48.809820+0000","last_undegraded":"2023-10-30T06:25:48.809820+0000","last_fullsized":"2023-10-30T06:25:48.809820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411352,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":584856,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.359999999999999,"5min":10.359999999999999,"15min":10.359999999999999},"min":{"1min":4.7409999999999997,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":24.350000000000001,"5min":24.350000000000001,"15min":24.350000000000001},"last":10.484999999999999},{"interface":"front","average":{"1min":12.083,"5min":12.083,"15min":12.083},"min":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"max":{"1min":46.798000000000002,"5min":46.798000000000002,"15min":46.798000000000002},"last":7.7880000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.798,"5min":13.798,"15min":13.798},"min":{"1min":5.5209999999999999,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":52.552,"5min":52.552,"15min":52.552},"last":11.411},{"interface":"front","average":{"1min":15.147,"5min":15.147,"15min":15.147},"min":{"1min":7.1589999999999998,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":27.946000000000002,"5min":27.946000000000002,"15min":27.946000000000002},"last":34.378999999999998}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":23.190999999999999,"5min":23.190999999999999,"15min":23.190999999999999},"min":{"1min":11.013,"5min":11.013,"15min":11.013},"max":{"1min":118.45699999999999,"5min":118.45699999999999,"15min":118.45699999999999},"last":14.32},{"interface":"front","average":{"1min":22.571999999999999,"5min":22.571999999999999,"15min":22.571999999999999},"min":{"1min":7.8739999999999997,"5min":7.8739999999999997,"15min":7.8739999999999997},"max":{"1min":124.075,"5min":124.075,"15min":124.075},"last":17.625}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":13.675000000000001,"5min":13.675000000000001,"15min":13.675000000000001},"min":{"1min":7.2359999999999998,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":51.649000000000001,"5min":51.649000000000001,"15min":51.649000000000001},"last":13.433999999999999},{"interface":"front","average":{"1min":17.998999999999999,"5min":17.998999999999999,"15min":17.998999999999999},"min":{"1min":7.9400000000000004,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":38.271999999999998,"5min":38.271999999999998,"15min":38.271999999999998},"last":32.146000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.052,"5min":21.052,"15min":21.052},"min":{"1min":10.776,"5min":10.776,"15min":10.776},"max":{"1min":123.754,"5min":123.754,"15min":123.754},"last":18.722999999999999},{"interface":"front","average":{"1min":23.079999999999998,"5min":23.079999999999998,"15min":23.079999999999998},"min":{"1min":8.5679999999999996,"5min":8.5679999999999996,"15min":8.5679999999999996},"max":{"1min":122.72499999999999,"5min":122.72499999999999,"15min":122.72499999999999},"last":9.1340000000000003}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.457000000000001,"5min":21.457000000000001,"15min":21.457000000000001},"min":{"1min":11.9,"5min":11.9,"15min":11.9},"max":{"1min":66.400999999999996,"5min":66.400999999999996,"15min":66.400999999999996},"last":19.664999999999999},{"interface":"front","average":{"1min":19.859000000000002,"5min":19.859000000000002,"15min":19.859000000000002},"min":{"1min":8.2070000000000007,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":53.149000000000001,"5min":53.149000000000001,"15min":53.149000000000001},"last":16.114999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.513999999999999,"5min":21.513999999999999,"15min":21.513999999999999},"min":{"1min":10.657,"5min":10.657,"15min":10.657},"max":{"1min":45.235999999999997,"5min":45.235999999999997,"15min":45.235999999999997},"last":30.931000000000001},{"interface":"front","average":{"1min":21.594999999999999,"5min":21.594999999999999,"15min":21.594999999999999},"min":{"1min":8.9930000000000003,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":69.991,"5min":69.991,"15min":69.991},"last":29.335999999999999}]}]},{"osd":6,"up_from":16,"seq":68719476762,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":14.706,"5min":14.706,"15min":14.706},"min":{"1min":4.8300000000000001,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":94.343000000000004,"5min":94.343000000000004,"15min":94.343000000000004},"last":10.247999999999999},{"interface":"front","average":{"1min":15.754,"5min":15.754,"15min":15.754},"min":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":122.054,"5min":122.054,"15min":122.054},"last":7.7389999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":10.816000000000001,"5min":10.816000000000001,"15min":10.816000000000001},"min":{"1min":6.4379999999999997,"5min":6.4379999999999997,"15min":6.4379999999999997},"max":{"1min":18.992999999999999,"5min":18.992999999999999,"15min":18.992999999999999},"last":11.891},{"interface":"front","average":{"1min":33.863999999999997,"5min":33.863999999999997,"15min":33.863999999999997},"min":{"1min":5.9269999999999996,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":387.15600000000001,"5min":387.15600000000001,"15min":387.15600000000001},"last":10.932}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.956,"5min":19.956,"15min":19.956},"min":{"1min":6.4359999999999999,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":156.90299999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":8.4909999999999997},{"interface":"front","average":{"1min":20.584,"5min":20.584,"15min":20.584},"min":{"1min":8.8940000000000001,"5min":8.8940000000000001,"15min":8.8940000000000001},"max":{"1min":143.578,"5min":143.578,"15min":143.578},"last":20.09}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.963000000000001,"5min":21.963000000000001,"15min":21.963000000000001},"min":{"1min":10.696999999999999,"5min":10.696999999999999,"15min":10.696999999999999},"max":{"1min":141.18799999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":17.905999999999999},{"interface":"front","average":{"1min":25.420999999999999,"5min":25.420999999999999,"15min":25.420999999999999},"min":{"1min":13.539999999999999,"5min":13.539999999999999,"15min":13.539999999999999},"max":{"1min":161.32900000000001,"5min":161.32900000000001,"15min":161.32900000000001},"last":22.198}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":15.045,"5min":15.045,"15min":15.045},"min":{"1min":6.9560000000000004,"5min":6.9560000000000004,"15min":6.9560000000000004},"max":{"1min":74.079999999999998,"5min":74.079999999999998,"15min":74.079999999999998},"last":9.4130000000000003},{"interface":"front","average":{"1min":15.702,"5min":15.702,"15min":15.702},"min":{"1min":9.2270000000000003,"5min":9.2270000000000003,"15min":9.2270000000000003},"max":{"1min":58.613999999999997,"5min":58.613999999999997,"15min":58.613999999999997},"last":12.499000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":17.966000000000001,"5min":17.966000000000001,"15min":17.966000000000001},"min":{"1min":9.6560000000000006,"5min":9.6560000000000006,"15min":9.6560000000000006},"max":{"1min":71.757000000000005,"5min":71.757000000000005,"15min":71.757000000000005},"last":16.027000000000001},{"interface":"front","average":{"1min":21.762,"5min":21.762,"15min":21.762},"min":{"1min":14.734,"5min":14.734,"15min":14.734},"max":{"1min":80.119,"5min":80.119,"15min":80.119},"last":21.306000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.741,"5min":19.741,"15min":19.741},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":81.247,"5min":81.247,"15min":81.247},"last":17.224},{"interface":"front","average":{"1min":19.088000000000001,"5min":19.088000000000001,"15min":19.088000000000001},"min":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":70.989000000000004,"5min":70.989000000000004,"15min":70.989000000000004},"last":13.051}]}]},{"osd":3,"up_from":16,"seq":68719476762,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":187016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":11.06,"5min":11.06,"15min":11.06},"min":{"1min":4.2679999999999998,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":24.466000000000001,"5min":24.466000000000001,"15min":24.466000000000001},"last":9.0879999999999992},{"interface":"front","average":{"1min":10.257999999999999,"5min":10.257999999999999,"15min":10.257999999999999},"min":{"1min":6,"5min":6,"15min":6},"max":{"1min":19.263999999999999,"5min":19.263999999999999,"15min":19.263999999999999},"last":9.8580000000000005}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":26.827000000000002,"5min":26.827000000000002,"15min":26.827000000000002},"min":{"1min":6.4020000000000001,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":165.76400000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":11.885},{"interface":"front","average":{"1min":28.079000000000001,"5min":28.079000000000001,"15min":28.079000000000001},"min":{"1min":8.5760000000000005,"5min":8.5760000000000005,"15min":8.5760000000000005},"max":{"1min":190.34200000000001,"5min":190.34200000000001,"15min":190.34200000000001},"last":24.646999999999998}]},{"osd":2,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":18.428000000000001,"5min":18.428000000000001,"15min":18.428000000000001},"min":{"1min":12.856999999999999,"5min":12.856999999999999,"15min":12.856999999999999},"max":{"1min":41.229999999999997,"5min":41.229999999999997,"15min":41.229999999999997},"last":30.335000000000001},{"interface":"front","average":{"1min":17.199000000000002,"5min":17.199000000000002,"15min":17.199000000000002},"min":{"1min":8.1739999999999995,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":33.060000000000002,"5min":33.060000000000002,"15min":33.060000000000002},"last":12.529999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.620000000000001,"5min":20.620000000000001,"15min":20.620000000000001},"min":{"1min":9.9420000000000002,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":42.274999999999999,"5min":42.274999999999999,"15min":42.274999999999999},"last":15.193},{"interface":"front","average":{"1min":11.614000000000001,"5min":11.614000000000001,"15min":11.614000000000001},"min":{"1min":6.7030000000000003,"5min":6.7030000000000003,"15min":6.7030000000000003},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":8.2959999999999994}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.658999999999999,"5min":19.658999999999999,"15min":19.658999999999999},"min":{"1min":7.4450000000000003,"5min":7.4450000000000003,"15min":7.4450000000000003},"max":{"1min":56.796999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":11.026},{"interface":"front","average":{"1min":20.798999999999999,"5min":20.798999999999999,"15min":20.798999999999999},"min":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"max":{"1min":58.576000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":13.747999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":29.138000000000002,"5min":29.138000000000002,"15min":29.138000000000002},"min":{"1min":11.694000000000001,"5min":11.694000000000001,"15min":11.694000000000001},"max":{"1min":169.05199999999999,"5min":169.05199999999999,"15min":169.05199999999999},"last":31.268999999999998},{"interface":"front","average":{"1min":15.494,"5min":15.494,"15min":15.494},"min":{"1min":9.6419999999999995,"5min":9.6419999999999995,"15min":9.6419999999999995},"max":{"1min":30.957000000000001,"5min":30.957000000000001,"15min":30.957000000000001},"last":13.132999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":17.986000000000001,"5min":17.986000000000001,"15min":17.986000000000001},"min":{"1min":8.5950000000000006,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":38.887999999999998,"5min":38.887999999999998,"15min":38.887999999999998},"last":14.632999999999999},{"interface":"front","average":{"1min":29.614999999999998,"5min":29.614999999999998,"15min":29.614999999999998},"min":{"1min":10.025,"5min":10.025,"15min":10.025},"max":{"1min":170.232,"5min":170.232,"15min":170.232},"last":23.940000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476760,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":187016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.809000000000001,"5min":27.809000000000001,"15min":27.809000000000001},"min":{"1min":5.0730000000000004,"5min":5.0730000000000004,"15min":5.0730000000000004},"max":{"1min":258.66199999999998,"5min":258.66199999999998,"15min":258.66199999999998},"last":7.6989999999999998},{"interface":"front","average":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"min":{"1min":6.016,"5min":6.016,"15min":6.016},"max":{"1min":203.38499999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":8.5269999999999992}]},{"osd":1,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":16.311,"5min":16.311,"15min":16.311},"min":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":45.936999999999998,"5min":45.936999999999998,"15min":45.936999999999998},"last":10.576000000000001},{"interface":"front","average":{"1min":14.839,"5min":14.839,"15min":14.839},"min":{"1min":6.8360000000000003,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":37.390999999999998,"5min":37.390999999999998,"15min":37.390999999999998},"last":9.5419999999999998}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.326000000000001,"5min":19.326000000000001,"15min":19.326000000000001},"min":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":183.38900000000001,"5min":183.38900000000001,"15min":183.38900000000001},"last":8.1720000000000006},{"interface":"front","average":{"1min":23.067,"5min":23.067,"15min":23.067},"min":{"1min":6.8810000000000002,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":189.374,"5min":189.374,"15min":189.374},"last":11.192}]},{"osd":4,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":21.959,"5min":21.959,"15min":21.959},"min":{"1min":7.7229999999999999,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":186.01400000000001,"5min":186.01400000000001,"15min":186.01400000000001},"last":13.452},{"interface":"front","average":{"1min":20.114999999999998,"5min":20.114999999999998,"15min":20.114999999999998},"min":{"1min":6.2400000000000002,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":167.67599999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":19.079000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.559999999999999,"5min":27.559999999999999,"15min":27.559999999999999},"min":{"1min":8.3040000000000003,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":215.80500000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":17.233000000000001},{"interface":"front","average":{"1min":27.331,"5min":27.331,"15min":27.331},"min":{"1min":8.6989999999999998,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":187.863,"5min":187.863,"15min":187.863},"last":12.547000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.262,"5min":27.262,"15min":27.262},"min":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":228.81200000000001,"5min":228.81200000000001,"15min":228.81200000000001},"last":18.288},{"interface":"front","average":{"1min":30.349,"5min":30.349,"15min":30.349},"min":{"1min":9.3710000000000004,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":275.92200000000003,"5min":275.92200000000003,"15min":275.92200000000003},"last":11.772}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":27.227,"5min":27.227,"15min":27.227},"min":{"1min":13.772,"5min":13.772,"15min":13.772},"max":{"1min":170.43600000000001,"5min":170.43600000000001,"15min":170.43600000000001},"last":14.446999999999999},{"interface":"front","average":{"1min":23.538,"5min":23.538,"15min":23.538},"min":{"1min":8.8360000000000003,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":157.00200000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":19.623999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476760,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":10.695,"5min":10.695,"15min":10.695},"min":{"1min":6.109,"5min":6.109,"15min":6.109},"max":{"1min":25.989999999999998,"5min":25.989999999999998,"15min":25.989999999999998},"last":20.556999999999999},{"interface":"front","average":{"1min":11.907999999999999,"5min":11.907999999999999,"15min":11.907999999999999},"min":{"1min":5.0010000000000003,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":30.788,"5min":30.788,"15min":30.788},"last":25.59}]},{"osd":2,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.052,"5min":20.052,"15min":20.052},"min":{"1min":6.5110000000000001,"5min":6.5110000000000001,"15min":6.5110000000000001},"max":{"1min":127.95399999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":19.131},{"interface":"front","average":{"1min":14.872,"5min":14.872,"15min":14.872},"min":{"1min":7.5720000000000001,"5min":7.5720000000000001,"15min":7.5720000000000001},"max":{"1min":35.834000000000003,"5min":35.834000000000003,"15min":35.834000000000003},"last":19.934000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.649999999999999,"5min":22.649999999999999,"15min":22.649999999999999},"min":{"1min":8.984,"5min":8.984,"15min":8.984},"max":{"1min":99.790999999999997,"5min":99.790999999999997,"15min":99.790999999999997},"last":22.321000000000002},{"interface":"front","average":{"1min":16.715,"5min":16.715,"15min":16.715},"min":{"1min":9.4689999999999994,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":26.454999999999998,"5min":26.454999999999998,"15min":26.454999999999998},"last":9.5980000000000008}]},{"osd":4,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":16.649999999999999,"5min":16.649999999999999,"15min":16.649999999999999},"min":{"1min":10.246,"5min":10.246,"15min":10.246},"max":{"1min":41.134999999999998,"5min":41.134999999999998,"15min":41.134999999999998},"last":8.6150000000000002},{"interface":"front","average":{"1min":25.373999999999999,"5min":25.373999999999999,"15min":25.373999999999999},"min":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"max":{"1min":103.705,"5min":103.705,"15min":103.705},"last":10.199999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":19.039000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"min":{"1min":12.388999999999999,"5min":12.388999999999999,"15min":12.388999999999999},"max":{"1min":43.847000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":11.302},{"interface":"front","average":{"1min":28.347999999999999,"5min":28.347999999999999,"15min":28.347999999999999},"min":{"1min":12.076000000000001,"5min":12.076000000000001,"15min":12.076000000000001},"max":{"1min":98.707999999999998,"5min":98.707999999999998,"15min":98.707999999999998},"last":18.292999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":20.504000000000001,"5min":20.504000000000001,"15min":20.504000000000001},"min":{"1min":8.6869999999999994,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":43.192999999999998,"5min":43.192999999999998,"15min":43.192999999999998},"last":55.631},{"interface":"front","average":{"1min":22.300999999999998,"5min":22.300999999999998,"15min":22.300999999999998},"min":{"1min":13.728,"5min":13.728,"15min":13.728},"max":{"1min":44.545000000000002,"5min":44.545000000000002,"15min":44.545000000000002},"last":54.656999999999996}]},{"osd":7,"last update":"Mon Oct 30 06:25:12 2023","interfaces":[{"interface":"back","average":{"1min":22.123999999999999,"5min":22.123999999999999,"15min":22.123999999999999},"min":{"1min":11.917999999999999,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":45.927999999999997,"5min":45.927999999999997,"15min":45.927999999999997},"last":23.265000000000001},{"interface":"front","average":{"1min":23.283000000000001,"5min":23.283000000000001,"15min":23.283000000000001},"min":{"1min":13.298,"5min":13.298,"15min":13.298},"max":{"1min":45.130000000000003,"5min":45.130000000000003,"15min":45.130000000000003},"last":28.651}]}]},{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":187016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":16.344999999999999,"5min":16.344999999999999,"15min":16.344999999999999},"min":{"1min":4.4249999999999998,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":131.477,"5min":131.477,"15min":131.477},"last":6.2309999999999999},{"interface":"front","average":{"1min":16.484000000000002,"5min":16.484000000000002,"15min":16.484000000000002},"min":{"1min":5.6340000000000003,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":129.386,"5min":129.386,"15min":129.386},"last":10.538}]},{"osd":1,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":10.994,"5min":10.994,"15min":10.994},"min":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"max":{"1min":25.774000000000001,"5min":25.774000000000001,"15min":25.774000000000001},"last":9.6289999999999996},{"interface":"front","average":{"1min":19.489000000000001,"5min":19.489000000000001,"15min":19.489000000000001},"min":{"1min":9.9960000000000004,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":91.295000000000002,"5min":91.295000000000002,"15min":91.295000000000002},"last":12.532}]},{"osd":2,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":18.219000000000001,"5min":18.219000000000001,"15min":18.219000000000001},"min":{"1min":6.7300000000000004,"5min":6.7300000000000004,"15min":6.7300000000000004},"max":{"1min":134.21199999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":18.957000000000001},{"interface":"front","average":{"1min":14.635,"5min":14.635,"15min":14.635},"min":{"1min":6.7759999999999998,"5min":6.7759999999999998,"15min":6.7759999999999998},"max":{"1min":27.780000000000001,"5min":27.780000000000001,"15min":27.780000000000001},"last":17.34}]},{"osd":3,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":20.669,"5min":20.669,"15min":20.669},"min":{"1min":7.0190000000000001,"5min":7.0190000000000001,"15min":7.0190000000000001},"max":{"1min":132.56100000000001,"5min":132.56100000000001,"15min":132.56100000000001},"last":8.7189999999999994},{"interface":"front","average":{"1min":21.187999999999999,"5min":21.187999999999999,"15min":21.187999999999999},"min":{"1min":6.5780000000000003,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":124.684,"5min":124.684,"15min":124.684},"last":13.532999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":13.888,"5min":13.888,"15min":13.888},"min":{"1min":6.9610000000000003,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":24.963000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":17.361999999999998},{"interface":"front","average":{"1min":15.602,"5min":15.602,"15min":15.602},"min":{"1min":8.3089999999999993,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":24.84,"5min":24.84,"15min":24.84},"last":14.364000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":20.725000000000001,"5min":20.725000000000001,"15min":20.725000000000001},"min":{"1min":8.3179999999999996,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":125.792,"5min":125.792,"15min":125.792},"last":16.236000000000001},{"interface":"front","average":{"1min":17.007000000000001,"5min":17.007000000000001,"15min":17.007000000000001},"min":{"1min":10.273,"5min":10.273,"15min":10.273},"max":{"1min":28.591999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":14.954000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:25:14 2023","interfaces":[{"interface":"back","average":{"1min":32.167999999999999,"5min":32.167999999999999,"15min":32.167999999999999},"min":{"1min":8.3480000000000008,"5min":8.3480000000000008,"15min":8.3480000000000008},"max":{"1min":327.29599999999999,"5min":327.29599999999999,"15min":327.29599999999999},"last":19.181999999999999},{"interface":"front","average":{"1min":33.918999999999997,"5min":33.918999999999997,"15min":33.918999999999997},"min":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"max":{"1min":323.82400000000001,"5min":323.82400000000001,"15min":323.82400000000001},"last":7.5499999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":584856,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":15.087,"5min":15.087,"15min":15.087},"min":{"1min":5.4829999999999997,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":68.513999999999996,"5min":68.513999999999996,"15min":68.513999999999996},"last":28.821999999999999},{"interface":"front","average":{"1min":17.041,"5min":17.041,"15min":17.041},"min":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"max":{"1min":88.816000000000003,"5min":88.816000000000003,"15min":88.816000000000003},"last":27.027999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":17.635000000000002,"5min":17.635000000000002,"15min":17.635000000000002},"min":{"1min":6.9269999999999996,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":87.515000000000001,"5min":87.515000000000001,"15min":87.515000000000001},"last":16.911000000000001},{"interface":"front","average":{"1min":18.013000000000002,"5min":18.013000000000002,"15min":18.013000000000002},"min":{"1min":6.21,"5min":6.21,"15min":6.21},"max":{"1min":85.832999999999998,"5min":85.832999999999998,"15min":85.832999999999998},"last":26.303000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":18.363,"5min":18.363,"15min":18.363},"min":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":113.943,"5min":113.943,"15min":113.943},"last":15.907},{"interface":"front","average":{"1min":15.470000000000001,"5min":15.470000000000001,"15min":15.470000000000001},"min":{"1min":6.5279999999999996,"5min":6.5279999999999996,"15min":6.5279999999999996},"max":{"1min":80.004000000000005,"5min":80.004000000000005,"15min":80.004000000000005},"last":17.914000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":21.841000000000001,"5min":21.841000000000001,"15min":21.841000000000001},"min":{"1min":7.3730000000000002,"5min":7.3730000000000002,"15min":7.3730000000000002},"max":{"1min":82.506,"5min":82.506,"15min":82.506},"last":14.419},{"interface":"front","average":{"1min":23.952000000000002,"5min":23.952000000000002,"15min":23.952000000000002},"min":{"1min":9.6720000000000006,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":91.183000000000007,"5min":91.183000000000007,"15min":91.183000000000007},"last":18.552}]},{"osd":4,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":12.738,"5min":12.738,"15min":12.738},"min":{"1min":7.0099999999999998,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":25.195,"5min":25.195,"15min":25.195},"last":19.675000000000001},{"interface":"front","average":{"1min":17.186,"5min":17.186,"15min":17.186},"min":{"1min":8.7100000000000009,"5min":8.7100000000000009,"15min":8.7100000000000009},"max":{"1min":59.609000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":7.2279999999999998}]},{"osd":6,"last update":"Mon Oct 30 06:25:17 2023","interfaces":[{"interface":"back","average":{"1min":16.122,"5min":16.122,"15min":16.122},"min":{"1min":8.6820000000000004,"5min":8.6820000000000004,"15min":8.6820000000000004},"max":{"1min":66.765000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":46.561999999999998},{"interface":"front","average":{"1min":21.257999999999999,"5min":21.257999999999999,"15min":21.257999999999999},"min":{"1min":11.551,"5min":11.551,"15min":11.551},"max":{"1min":71.771000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":39.637999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:25:11 2023","interfaces":[{"interface":"back","average":{"1min":19.148,"5min":19.148,"15min":19.148},"min":{"1min":9.8770000000000007,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":64.677000000000007,"5min":64.677000000000007,"15min":64.677000000000007},"last":25.045999999999999},{"interface":"front","average":{"1min":18.381,"5min":18.381,"15min":18.381},"min":{"1min":10.464,"5min":10.464,"15min":10.464},"max":{"1min":42.531999999999996,"5min":42.531999999999996,"15min":42.531999999999996},"last":21.984000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":187016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":9.3100000000000005,"5min":9.3100000000000005,"15min":9.3100000000000005},"min":{"1min":4.5410000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":14.558,"5min":14.558,"15min":14.558},"last":7.282},{"interface":"front","average":{"1min":10.558,"5min":10.558,"15min":10.558},"min":{"1min":6.2850000000000001,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":14.507999999999999,"5min":14.507999999999999,"15min":14.507999999999999},"last":10.012}]},{"osd":1,"last update":"Mon Oct 30 06:25:15 2023","interfaces":[{"interface":"back","average":{"1min":11.471,"5min":11.471,"15min":11.471},"min":{"1min":6.2960000000000003,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":19.382999999999999,"5min":19.382999999999999,"15min":19.382999999999999},"last":9.4329999999999998},{"interface":"front","average":{"1min":14.153,"5min":14.153,"15min":14.153},"min":{"1min":6.609,"5min":6.609,"15min":6.609},"max":{"1min":26.864000000000001,"5min":26.864000000000001,"15min":26.864000000000001},"last":30.751999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.968,"5min":11.968,"15min":11.968},"min":{"1min":6.6200000000000001,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":16.492999999999999,"5min":16.492999999999999,"15min":16.492999999999999},"last":15.032},{"interface":"front","average":{"1min":15.776,"5min":15.776,"15min":15.776},"min":{"1min":12.058,"5min":12.058,"15min":12.058},"max":{"1min":21.231000000000002,"5min":21.231000000000002,"15min":21.231000000000002},"last":11.225}]},{"osd":3,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":11.265000000000001,"5min":11.265000000000001,"15min":11.265000000000001},"min":{"1min":5.6689999999999996,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":29.881,"5min":29.881,"15min":29.881},"last":11.885999999999999},{"interface":"front","average":{"1min":16.065999999999999,"5min":16.065999999999999,"15min":16.065999999999999},"min":{"1min":9.8130000000000006,"5min":9.8130000000000006,"15min":9.8130000000000006},"max":{"1min":30.285,"5min":30.285,"15min":30.285},"last":16.196000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":19.603000000000002,"5min":19.603000000000002,"15min":19.603000000000002},"min":{"1min":6.4950000000000001,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":64.474999999999994,"5min":64.474999999999994,"15min":64.474999999999994},"last":10.619999999999999},{"interface":"front","average":{"1min":16.212,"5min":16.212,"15min":16.212},"min":{"1min":5.7770000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":75.980000000000004,"5min":75.980000000000004,"15min":75.980000000000004},"last":8.2919999999999998}]},{"osd":6,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.779,"5min":18.779,"15min":18.779},"min":{"1min":16.312000000000001,"5min":16.312000000000001,"15min":16.312000000000001},"max":{"1min":26.965,"5min":26.965,"15min":26.965},"last":17.841999999999999},{"interface":"front","average":{"1min":14.026,"5min":14.026,"15min":14.026},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":25.664000000000001,"5min":25.664000000000001,"15min":25.664000000000001},"last":12.475}]},{"osd":7,"last update":"Mon Oct 30 06:25:13 2023","interfaces":[{"interface":"back","average":{"1min":18.358000000000001,"5min":18.358000000000001,"15min":18.358000000000001},"min":{"1min":3.5579999999999998,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":74.977999999999994,"5min":74.977999999999994,"15min":74.977999999999994},"last":15.619},{"interface":"front","average":{"1min":22.355,"5min":22.355,"15min":22.355},"min":{"1min":5.0060000000000002,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":83.138000000000005,"5min":83.138000000000005,"15min":83.138000000000005},"last":18.872}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:25:59.104 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:25:59.105 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:25:59.105 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.106 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:25:59.138 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:59.137+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:59.239 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:25:59.238+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:25:59.284 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.284 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:25:59.340 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:59.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:59.420 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.420 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:25:59.440 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:25:59.439+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:25:59.542 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:25:59.540+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:25:59.556 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.557 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:25:59.642 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:59.641+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:25:59.732 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.733 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:25:59.743 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:59.741+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:59.854 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:25:59.852+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:25:59.867 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:25:59.867 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:25:59.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:25:59.945 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:25:59.944+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.046 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.047 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:00.045+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:00.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:00.147 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:00.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:00.186 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.186 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.192 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:55.379 86065== 2023-10-30T06:26:00.192 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:55.379 86065== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:26:00.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:00.202 INFO:tasks.ceph.mon.b.smithi112.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:26:00.252 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:00.250+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:00.349 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:00.348+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:00.361 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.361 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:00.450 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:00.448+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.497 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:00.551 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:00.550+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:00.652 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:00.651+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.674 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:00.770 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:00.768+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:00.854 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:00.854 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:00.855 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:00.854+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:00.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:00.956 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:00.954+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:01.032 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:01.032 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:01.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:01.085 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:01.083+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:01.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:01.162 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:01.160+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:01.168 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:01.168 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:01.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:01.263 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:01.261+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:01.360 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:01.358+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:01.361 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:01.361 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:01.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:01.375 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:01.376 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.458 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:01.459 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.460 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:01.459+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:01.515 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:01.515 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:01.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:01.548 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:01.548 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.573 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:01.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:01.633 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:01.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.662 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:01.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:01.665 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:01.666 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:01.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:01.723 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:01.724 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.762 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:01.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:01.809 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-30T06:26:01.850 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:01.850 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:01.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:01.851 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:01.851 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:01.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:01.867 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:01.865+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:01.900 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:01.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:01.967 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:01.966+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:02.020 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:02.020 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:02.059 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:02.059 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:02.059 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:02.059 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:02.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:02.072 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:02.072+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:02.169 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:02.168+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:02.235 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:02.236 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:02.236 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:02.236 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:02.236 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:02.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:02.289 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:02.287+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:02.372 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:02.371+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:02.412 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:02.412 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:02.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:02.413 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:02.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.413 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:02.414 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:02.414 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.257181143, 2023-10-30T06:26:02.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:02.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:02.415 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:02.415 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:02.415 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:02.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:02.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.416 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:02.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:02.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:02.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:02.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:02.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:02.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:02.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:02.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:02.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:02.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:02.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.425 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.426 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.426 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:02.426 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:02.427 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 107.34400728, 2023-10-30T06:26:02.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:02.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.427 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:02.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:02.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:02.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:02.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:02.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:02.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:02.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:02.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:02.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:02.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:02.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:02.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:02.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.437 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.438 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:02.438 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:02.439 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.683829311, 2023-10-30T06:26:02.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:02.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.439 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:02.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:02.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:02.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:02.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:02.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:02.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:02.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:02.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:02.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:02.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:02.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:02.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.449 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.450 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.450 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:02.450 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:02.451 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.608406104, 2023-10-30T06:26:02.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:02.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.451 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:02.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:02.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:02.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:02.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:02.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:02.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:02.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:02.461 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.461 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.461 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.337282973, 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:02.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.463 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:02.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:02.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:02.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:02.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:02.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:02.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:02.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:02.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:02.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:02.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:02.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.472 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:02.473 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.307003263, 2023-10-30T06:26:02.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:02.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.474 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:02.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:02.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:02.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:02.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:02.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:02.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:02.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:02.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:02.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:02.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:02.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:02.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.484 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.933092179, 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.486 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:02.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:02.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:02.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:02.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:02.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:02.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:02.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:02.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:02.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:02.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.495 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.496 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.496 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:02.496 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.928593478, 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:02.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:02.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:02.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:02.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:02.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:02.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:02.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:02.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:02.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.636989539, 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:02.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:02.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:02.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:02.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:02.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:02.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:02.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:02.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.519 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.519 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:02.519 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:02.520 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.63105222199999, 2023-10-30T06:26:02.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:02.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.520 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:02.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:02.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:02.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:02.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:02.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:02.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:02.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:02.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:02.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:02.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:02.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:02.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:02.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:02.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:02.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.530 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.62975320699999, 2023-10-30T06:26:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:02.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:02.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:02.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:02.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:02.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:02.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:02.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:02.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:02.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:02.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:02.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:02.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.542 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.543 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:02.543 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:02.543 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.526764399, 2023-10-30T06:26:02.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:02.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.544 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:02.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:02.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:02.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:02.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:02.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:02.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:02.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:02.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:02.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:02.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:02.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.554 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:02.554 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:02.554 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.254861132, 2023-10-30T06:26:02.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:02.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.555 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:02.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:02.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:02.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:02.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:02.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:02.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:02.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:02.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:02.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.565 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.566 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:02.566 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:02.566 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.24736294, 2023-10-30T06:26:02.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:02.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:02.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:02.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:02.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:02.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:02.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:02.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:02.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:02.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:02.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:02.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:02.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.577 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:02.577 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:02.577 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.246701735, 2023-10-30T06:26:02.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:02.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.578 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:02.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:02.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:02.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:02.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:02.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:02.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:02.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:02.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:02.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:02.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:02.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.588 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.234115897, 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.590 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:02.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:02.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:02.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:02.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:02.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:02.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:02.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:02.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:02.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:02.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:02.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:02.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:02.599 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.599 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.599 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.600 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:02.600 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:02.600 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.22802368399999, 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:02.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:02.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:02.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:02.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:02.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:02.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:02.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:02.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:02.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:02.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:02.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.611 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 105.227028343, 2023-10-30T06:26:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.613 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:02.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:02.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:02.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:02.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:02.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:02.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:02.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:02.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:02.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:02.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:02.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:02.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:02.622 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.622 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.622 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 97.904355772000002, 2023-10-30T06:26:02.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:02.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.624 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:02.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:02.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:02.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:02.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:02.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:02.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:02.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:02.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.633 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:02.634 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:02.635 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 97.870914287000005, 2023-10-30T06:26:02.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:02.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:02.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:02.635 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:02.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:02.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:02.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:02.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:02.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:02.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:02.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:02.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:02.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:02.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:02.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:02.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:02.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:02.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:02.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:02.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.645 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:02.645 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:02.646 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:02.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:02.648 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:02.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:02.748 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:02.748+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:02.815 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:02.815 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:02.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:02.849 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:02.848+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:02.950 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:02.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:02.995 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:02.996 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:03.006 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:03.052 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:03.051+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:03.133 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:03.134 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:03.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:03.135 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:03.135 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:03.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:03.153 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:03.152+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:03.253 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:03.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:03.309 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:03.309 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:03.309 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:03.309 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:03.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:03.355 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:03.354+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:03.443 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:03.444 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:03.444 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:03.444 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:03.444 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:03.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:03.455 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:03.455+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:03.556 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:03.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:03.580 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:03.581 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:03.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:03.581 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:03.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.582 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:03.582 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:03.582 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 26.608182683999999, 2023-10-30T06:26:03.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:03.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:03.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:03.583 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:03.583 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:03.583 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:03.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:03.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.584 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:03.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:03.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:03.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:03.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:03.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:03.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:03.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:03.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:03.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:03.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:03.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:03.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:03.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:03.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:03.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:03.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:03.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:03.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:03.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:03.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:03.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:03.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:03.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:03.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:03.592 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:03.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:03.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:03.592 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:03.593 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:03.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:03.657 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:03.656+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:03.728 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:03.728 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:03.728 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:03.728 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:03.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:03.758 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:03.758+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:03.859 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:03.858+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:03.876 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:26:03.877 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-30T06:26:03.877 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T06:26:03.878 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:26:03.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:03.908 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:03.908 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:03.908 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:03.908 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:03.909 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:03.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:03.959 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:03.959+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:04.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:04.060 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:04.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:04.062 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:26:04.062 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:26:04.062 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2023-10-30T06:26:04.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:26:04.086 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:04.087 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:04.096 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:04.161 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:04.161+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:04.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-10-30T06:26:04.186+0000 7fd0bdf66640 0 --2- 172.21.15.79:0/115460066 >> [v2:172.21.15.79:3301/0,v1:172.21.15.79:6790/0] conn(0x7fd0b80aeba0 0x7fd0b813f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-10-30T06:26:04.262 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:04.262+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:04.277 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:04.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:04.278 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:04.278 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:04.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:04.364 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:04.363+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:04.417 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:04.417 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:04.418 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:04.418 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:04.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:04.465 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:04.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:04.566 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:04.566+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:04.656 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:04.656 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:04.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:04.656 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:04.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.657 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:04.657 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:04.657 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 111.315237792, 2023-10-30T06:26:04.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:04.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:04.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:04.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:04.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:04.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:04.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:04.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:04.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:04.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:04.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:04.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:04.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:04.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:04.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:04.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:04.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:04.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:04.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:04.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:04.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:04.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:04.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:04.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:04.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:04.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:04.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.675 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.675 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:04.676 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.994082601, 2023-10-30T06:26:04.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:04.677 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.677 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.677 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.678 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:04.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.678 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:04.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:04.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:04.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:04.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:04.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:04.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:04.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:04.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:04.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:04.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:04.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:04.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:04.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:04.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:04.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:04.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:04.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:04.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:04.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:04.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:04.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:04.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:04.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.989838044, 2023-10-30T06:26:04.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:04.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:04.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:04.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:04.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:04.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:04.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:04.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:04.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:04.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:04.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:04.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:04.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:04.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:04.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:04.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.904379619, 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:04.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.710 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:04.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.711 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:04.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:04.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:04.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:04.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:04.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:04.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:04.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:04.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:04.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:04.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:04.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:04.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:04.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:04.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.635136403, 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:04.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:04.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:04.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:04.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:04.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:04.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:04.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:04.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:04.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:04.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.739 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.739 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.634304992, 2023-10-30T06:26:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.740 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.633637865, 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:04.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:04.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:04.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:04.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:04.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:04.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.611363794, 2023-10-30T06:26:04.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:04.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:04.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:04.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:04.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:04.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:04.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.610715938, 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:04.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:04.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:04.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:04.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:04.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:04.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 108.610029865, 2023-10-30T06:26:04.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:04.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:04.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:04.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:04.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:04.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:04.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:04.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:04.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.53914789700001, 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:04.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.538483931, 2023-10-30T06:26:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:04.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:04.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:04.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:04.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:04.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:04.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:04.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:04.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:04.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:04.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:04.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.510030122, 2023-10-30T06:26:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:04.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:04.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:04.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.16826362899999, 2023-10-30T06:26:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:04.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:04.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:04.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.878 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.167514959, 2023-10-30T06:26:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.166846576, 2023-10-30T06:26:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.15699590200001, 2023-10-30T06:26:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.15632208700001, 2023-10-30T06:26:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.15567370399999, 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 100.151557238, 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:04.970 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:04.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:04.973 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:04.974 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:04.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:05.073 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:05.073+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:05.140 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:05.141 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:05.173 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:05.173+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:05.275 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:05.274+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:05.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:05.278 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:05.279 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:05.376 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:05.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:05.417 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:05.477 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:05.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:05.552 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.552 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:05.553 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:05.553 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:05.578 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:05.578+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:05.679 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:05.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:05.727 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.728 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:05.728 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:05.728 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:05.728 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:05.780 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:05.780+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:05.864 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:05.864 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:05.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 29.572266021000001, 2023-10-30T06:26:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:05.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:05.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:05.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:05.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:05.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:05.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:05.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:05.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:05.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:05.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:05.877 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:05.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:05.881 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:05.879+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:05.982 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:05.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:06.022 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:06.022 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:06.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:06.119 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:06.116+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:06.184 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:06.183+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:06.203 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:06.204 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:06.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:06.297 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:06.295+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:06.377 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:06.378 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:06.385 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:06.383+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:06.386 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:06.486 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:06.484+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:06.556 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:06.557 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:06.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:06.607 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:06.605+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:06.687 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:06.687+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:06.732 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:06.733 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:06.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:06.788 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:06.788+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:06.890 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:06.888+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:06.908 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:06.908 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:06.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:06.909 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:06.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.909 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:06.909 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:06.910 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.75259615, 2023-10-30T06:26:06.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:06.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:06.911 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:06.911 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:06.911 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:06.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:06.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.912 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:06.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:06.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:06.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:06.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:06.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:06.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:06.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:06.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:06.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:06.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:06.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:06.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:06.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:06.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:06.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:06.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:06.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.922 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.923 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:06.923 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:06.923 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 111.839422287, 2023-10-30T06:26:06.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:06.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.924 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:06.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:06.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:06.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:06.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:06.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:06.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:06.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:06.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:06.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:06.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:06.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:06.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:06.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:06.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:06.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:06.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:06.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:06.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.936 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.937 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:06.937 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:06.937 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 111.179244318, 2023-10-30T06:26:06.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:06.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.938 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:06.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:06.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:06.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:06.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:06.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:06.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:06.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:06.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:06.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:06.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:06.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:06.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:06.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:06.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:06.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:06.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:06.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:06.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.949 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.949 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.950 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:06.950 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:06.950 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 111.103821111, 2023-10-30T06:26:06.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:06.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.951 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:06.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:06.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:06.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:06.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:06.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:06.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:06.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:06.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:06.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:06.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:06.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:06.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:06.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:06.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:06.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:06.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:06.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:06.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:06.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:06.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.965 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.965 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.965 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:06.966 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:06.966 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.83269798000001, 2023-10-30T06:26:06.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:06.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.967 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:06.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:06.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:06.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:06.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:06.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:06.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:06.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:06.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:06.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:06.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:06.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:06.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:06.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:06.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:06.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:06.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:06.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:06.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:06.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:06.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.978 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.979 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:06.979 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:06.979 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.80241827, 2023-10-30T06:26:06.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:06.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.980 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.982 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:06.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:06.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:06.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:06.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:06.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:06.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:06.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:06.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:06.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:06.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:06.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:06.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:06.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:06.990 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.990 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:06.991 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.428507186, 2023-10-30T06:26:06.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:06.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:06.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:06.992 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:06.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:06.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:06.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:06.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:06.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:06.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:06.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:06.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:06.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:06.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:06.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:06.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:06.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:06.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:06.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:06.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:06.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:06.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:06.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:06.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:07.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:07.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:07.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:07.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:07.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:07.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:07.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.003 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.004 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:07.004 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:07.004 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.424008485, 2023-10-30T06:26:07.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:07.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.005 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:07.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:07.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:07.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:07.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:07.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:07.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:07.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:07.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:07.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:07.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:07.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:07.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:07.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:07.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.015 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:07.016 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:07.016 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.132404546, 2023-10-30T06:26:07.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:07.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.017 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:07.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:07.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:07.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:07.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:07.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:07.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:07.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:07.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:07.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:07.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:07.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:07.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:07.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:07.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:07.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:07.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:07.029 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.029 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.029 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.030 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:07.030 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:07.030 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.126467229, 2023-10-30T06:26:07.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:07.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.031 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:07.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:07.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:07.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:07.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:07.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:07.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:07.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:07.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:07.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:07.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:07.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:07.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:07.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:07.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:07.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:07.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.125168214, 2023-10-30T06:26:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:07.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.043 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:07.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:07.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:07.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:07.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:07.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:07.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:07.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:07.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:07.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:07.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:07.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:07.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:07.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:07.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.053 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.054 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:07.054 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:07.054 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 110.02217940600001, 2023-10-30T06:26:07.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:07.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.055 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:07.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:07.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:07.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:07.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:07.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:07.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:07.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:07.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:07.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:07.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.75027613899999, 2023-10-30T06:26:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.74277794699999, 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.74211674199999, 2023-10-30T06:26:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.729530904, 2023-10-30T06:26:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.723438691, 2023-10-30T06:26:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 109.72244335000001, 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.39977077899999, 2023-10-30T06:26:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 102.366329294, 2023-10-30T06:26:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:07.164 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.165 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:07.166 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:07.165+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:07.267 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:07.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:07.333 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:07.334 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:07.367 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:07.367+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:07.468 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:07.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:07.508 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:07.508 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:07.508 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:07.509 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:07.509 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:07.569 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:07.569+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:07.671 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:07.670+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:07.686 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:07.686 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:07.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:07.687 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:07.687 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:07.772 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:07.771+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:07.822 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:07.822 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:07.822 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:07.823 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:07.873 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:07.873+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:07.959 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:07.960 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:07.960 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:07.960 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:07.960 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:07.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:07.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:08.014 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:08.012+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:08.076 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:08.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:08.099 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:08.099 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:08.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:08.099 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:08.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.100 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:08.100 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:08.100 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 31.126676056000001, 2023-10-30T06:26:08.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:08.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:08.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:08.101 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:08.101 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:08.101 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:08.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:08.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:08.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:08.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:08.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:08.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:08.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:08.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:08.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:08.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:08.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:08.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:08.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:08.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:08.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:08.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:08.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:08.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:08.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:08.110 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:08.110 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:08.110 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:08.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:08.111 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:08.111 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:08.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:08.180 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:08.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:08.241 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:08.241 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:08.241 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:08.241 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:08.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:08.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-30T06:26:08.277 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:08.276+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:08.378 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:08.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:08.418 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:08.418 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:08.418 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:08.418 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:08.419 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:08.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:08.490 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:08.489+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:08.556 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:08.556 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:08.557 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:08.557 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:08.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:08.579 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:08.578+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:08.680 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:08.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:08.691 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:08.692 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:08.692 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:08.692 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:08.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:08.781 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:08.780+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:08.827 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:08.828 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:08.828 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:08.828 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:08.828 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:08.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:08.921 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:08.920+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:08.983 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:08.981+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:09.060 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:09.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:09.060 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:09.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.061 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:09.061 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:09.061 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 115.72370865800001, 2023-10-30T06:26:09.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:09.061 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.062 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.062 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.062 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:09.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.063 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:09.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:09.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:09.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:09.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:09.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:09.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:09.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:09.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:09.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:09.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:09.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:09.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:09.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:09.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:09.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:09.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:09.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.080 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.081 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.081 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.081 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:09.082 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.402553467, 2023-10-30T06:26:09.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:09.082 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.082 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.083 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.083 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:09.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.083 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:09.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:09.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:09.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:09.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:09.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:09.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:09.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:09.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:09.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:09.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:09.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:09.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:09.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:09.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:09.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:09.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:09.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:09.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.096 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:09.097 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.39830891, 2023-10-30T06:26:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:09.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:09.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:09.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:09.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:09.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:09.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:09.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:09.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:09.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:09.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:09.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:09.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:09.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:09.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:09.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:09.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:09.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:09.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.312850485, 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:09.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.114 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.114 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:09.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.115 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:09.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:09.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:09.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:09.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:09.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:09.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:09.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:09.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:09.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:09.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:09.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:09.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:09.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:09.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:09.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:09.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.128 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.128 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.129 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:09.129 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.04360726900001, 2023-10-30T06:26:09.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:09.129 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.130 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:09.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:09.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:09.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:09.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:09.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:09.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:09.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:09.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:09.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:09.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:09.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:09.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:09.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:09.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:09.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:09.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:09.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.144 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.144 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:09.144 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.042775858, 2023-10-30T06:26:09.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:09.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.145 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.145 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:09.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:09.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:09.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:09.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:09.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:09.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:09.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:09.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:09.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:09.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:09.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:09.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:09.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:09.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:09.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:09.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:09.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:09.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.159 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.159 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.042108731, 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:09.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:09.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:09.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:09.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:09.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:09.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:09.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:09.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:09.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:09.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:09.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:09.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:09.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:09.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:09.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:09.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:09.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:09.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:09.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.01983466, 2023-10-30T06:26:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:09.175 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.176 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:09.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:09.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:09.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:09.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:09.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:09.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:09.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:09.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:09.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:09.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:09.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:09.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:09.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:09.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:09.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:09.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:09.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:09.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:09.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:09.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.190 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.190 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:09.191 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.019186804, 2023-10-30T06:26:09.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:09.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.191 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.191 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.192 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:09.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.192 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:09.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:09.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:09.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:09.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:09.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:09.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:09.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:09.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:09.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:09.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:09.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:09.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:09.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:09.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:09.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:09.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:09.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:09.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:09.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:09.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.205 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:09.206 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.018500731, 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.207 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.208 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:09.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.208 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:09.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:09.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:09.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:09.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:09.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:09.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:09.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:09.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:09.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:09.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:09.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:09.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:09.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:09.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:09.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:09.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:09.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:09.222 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 111.94761876299999, 2023-10-30T06:26:09.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.223 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:09.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:09.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:09.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:09.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:09.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:09.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:09.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:09.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:09.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:09.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:09.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:09.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:09.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:09.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:09.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:09.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:09.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:09.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:09.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:09.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:09.237 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:09.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 111.946954797, 2023-10-30T06:26:09.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:09.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.238 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.238 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:09.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.239 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:09.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:09.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:09.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:09.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:09.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:09.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:09.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:09.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:09.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:09.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:09.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:09.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:09.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:09.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:09.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.253 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:09.253 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:09.253 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 111.91850098800001, 2023-10-30T06:26:09.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:09.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.254 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.254 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.254 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:09.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:09.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:09.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:09.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:09.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:09.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:09.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:09.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.576734495, 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.270 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:09.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.575985825, 2023-10-30T06:26:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.575317442, 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:09.300 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.301 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:09.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:09.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.56546676799999, 2023-10-30T06:26:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.56479295299999, 2023-10-30T06:26:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.56414457, 2023-10-30T06:26:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 104.560028104, 2023-10-30T06:26:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:09.377 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:09.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:09.380 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:09.378+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:09.480 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:09.478+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:09.548 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:09.549 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:09.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:09.580 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:09.579+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:09.682 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:09.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:09.684 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:09.685 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:09.685 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:09.685 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:09.685 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:09.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:09.783 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:09.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:09.819 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:09.820 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:09.820 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:09.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:09.883 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:09.883+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:09.955 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:09.955 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:09.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:10.007 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:10.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:10.087 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:10.086+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:10.129 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:10.129 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:10.130 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:10.130 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:10.130 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:10.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:10.188 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:10.187+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:10.269 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:10.269 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:10.270 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.977544277, 2023-10-30T06:26:10.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:10.271 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:10.271 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:10.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:10.271 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:10.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:10.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:10.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:10.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:10.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:10.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:10.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:10.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:10.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:10.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:10.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:10.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:10.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:10.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:10.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:10.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:10.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:10.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:10.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:10.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:10.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:10.281 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:10.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:10.322 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:10.322+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:10.389 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:10.388+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:10.420 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:10.420 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:10.420 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:10.420 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:10.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:10.515 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:10.513+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:10.592 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:10.590+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:10.624 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:10.625 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:10.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:10.693 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:10.693+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:10.794 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:10.793+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:10.799 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:10.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:10.800 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:10.800 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:10.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:10.898 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:10.895+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:10.974 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:10.975 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:10.975 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:10.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:11.027 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:11.026+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:11.096 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:11.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:11.111 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:11.111 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:11.112 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:11.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:11.206 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:11.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:11.331 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:11.331 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.332 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.172148619, 2023-10-30T06:26:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:11.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:11.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:11.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:11.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:11.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:11.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:11.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:11.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:11.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:11.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:11.344 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.345 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 116.258974756, 2023-10-30T06:26:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:11.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:11.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:11.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:11.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:11.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:11.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:11.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:11.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.356 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.357 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.598796787, 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.358 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:11.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:11.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:11.369 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.52337358, 2023-10-30T06:26:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.252250449, 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:11.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:11.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:11.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:11.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.392 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.221970739, 2023-10-30T06:26:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:11.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:11.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:11.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:11.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:11.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:11.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:11.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:11.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:11.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:11.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.404 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.848059655, 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.405 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:11.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:11.416 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.843560954, 2023-10-30T06:26:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.551957015, 2023-10-30T06:26:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:11.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:11.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:11.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:11.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.54601969799999, 2023-10-30T06:26:11.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:11.452 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.54472068299999, 2023-10-30T06:26:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.464 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:11.464 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:11.464 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.441731875, 2023-10-30T06:26:11.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.169828608, 2023-10-30T06:26:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:11.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:11.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:11.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:11.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:11.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.162330416, 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:11.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:11.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:11.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:11.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.498 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:11.499 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.161669211, 2023-10-30T06:26:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:11.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:11.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:11.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:11.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:11.509 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:11.510 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.149083373, 2023-10-30T06:26:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.511 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.521 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:11.522 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.14299115999999, 2023-10-30T06:26:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 114.141995819, 2023-10-30T06:26:11.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:11.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:11.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:11.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:11.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:11.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:11.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:11.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:11.544 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:11.545 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.819323248, 2023-10-30T06:26:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.546 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:11.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:11.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:11.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:11.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:11.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:11.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:11.556 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:11.557 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 106.78588176300001, 2023-10-30T06:26:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:11.558 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:11.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:11.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:11.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:11.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:11.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:11.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.568 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:11.569 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:11.569 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:11.570 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:11.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:11.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:11.572 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:11.570+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:11.672 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:11.671+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:11.738 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:11.739 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:11.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:11.772 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:11.772+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:11.873 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:11.872+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:11.917 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:11.917 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:11.918 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:11.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:11.974 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:11.974+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:12.075 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:12.074+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:12.095 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:12.095 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:12.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:12.096 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:12.096 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:12.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:12.176 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:12.175+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:12.271 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:12.271 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:12.272 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:12.272 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:12.277 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:12.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:12.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:12.378 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:12.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:12.408 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:12.408 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:12.408 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:12.408 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:12.409 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:12.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:12.479 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:12.477+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:12.546 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:12.546 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:12.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:12.546 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:12.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.547 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:12.547 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:12.547 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 35.573682609999999, 2023-10-30T06:26:12.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:12.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:12.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:12.548 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:12.548 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:12.549 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:12.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:12.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.549 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:12.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:12.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:12.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:12.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:12.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:12.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:12.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:12.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:12.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:12.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:12.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:12.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:12.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:12.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:12.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:12.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:12.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:12.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:12.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:12.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:12.557 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:12.557 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:12.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:12.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:12.558 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:12.558 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:12.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:12.601 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:12.599+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:12.681 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:12.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:12.697 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:12.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:12.781 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:12.780+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:12.834 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:12.835 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:12.835 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:12.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:12.883 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:12.882+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:12.969 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:12.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:12.970 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:12.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:12.983 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:12.982+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:13.084 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:13.083+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:13.107 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:13.107 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:13.108 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:13.108 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:13.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:13.185 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:13.184+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:13.282 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:13.282 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:13.283 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:13.283 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:13.285 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:13.284+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:13.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:13.387 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:13.385+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:13.515 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:13.516 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:13.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:13.516 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:13.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:13.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:13.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.17671126400001, 2023-10-30T06:26:13.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:13.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:13.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:13.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:13.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:13.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:13.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:13.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:13.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:13.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:13.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:13.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:13.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:13.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:13.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:13.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:13.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:13.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:13.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:13.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:13.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:13.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:13.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:13.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:13.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:13.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:13.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.534 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:13.535 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.855556073, 2023-10-30T06:26:13.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:13.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.535 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.536 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.536 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:13.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:13.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:13.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:13.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:13.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:13.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:13.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:13.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:13.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:13.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:13.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:13.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:13.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:13.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:13.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:13.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:13.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:13.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:13.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:13.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.550 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.550 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:13.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.851311516, 2023-10-30T06:26:13.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.552 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:13.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.552 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:13.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:13.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:13.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:13.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:13.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:13.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:13.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:13.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:13.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:13.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:13.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:13.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:13.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:13.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:13.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:13.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:13.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:13.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:13.566 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.765853091, 2023-10-30T06:26:13.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:13.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:13.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:13.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:13.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:13.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:13.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:13.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:13.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:13.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:13.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:13.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:13.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:13.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:13.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:13.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:13.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:13.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.581 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.581 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:13.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.496609875, 2023-10-30T06:26:13.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:13.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:13.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:13.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:13.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:13.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:13.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:13.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:13.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:13.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:13.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:13.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:13.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:13.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:13.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:13.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:13.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:13.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.596 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:13.596 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.495778464, 2023-10-30T06:26:13.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:13.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.597 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:13.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:13.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:13.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.495111337, 2023-10-30T06:26:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:13.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:13.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:13.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:13.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.625 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.472837266, 2023-10-30T06:26:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.626 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.627 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.640 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.47218941, 2023-10-30T06:26:13.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.642 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:13.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:13.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:13.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:13.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:13.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.655 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:13.656 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.471503337, 2023-10-30T06:26:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.657 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 116.40062136900001, 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.672 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:13.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:13.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:13.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:13.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:13.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:13.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 116.399957403, 2023-10-30T06:26:13.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.688 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:13.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:13.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:13.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:13.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:13.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:13.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:13.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:13.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:13.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:13.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:13.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.702 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:13.702 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:13.702 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 116.371503594, 2023-10-30T06:26:13.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:13.703 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.703 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:13.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:13.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:13.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:13.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:13.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:13.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:13.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:13.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:13.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:13.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:13.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:13.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:13.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:13.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.02973710099999, 2023-10-30T06:26:13.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:13.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:13.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:13.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:13.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:13.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:13.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:13.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:13.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:13.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:13.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:13.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:13.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:13.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:13.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.028988431, 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.732 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.733 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:13.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:13.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:13.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:13.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:13.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:13.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:13.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:13.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:13.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:13.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:13.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:13.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:13.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:13.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:13.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.028320048, 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:13.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:13.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:13.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:13.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:13.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:13.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:13.760 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.761 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.01846937400001, 2023-10-30T06:26:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.762 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:13.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:13.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:13.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:13.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.775 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:13.776 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.01779555900001, 2023-10-30T06:26:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.777 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:13.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:13.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:13.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:13.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:13.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.790 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:13.791 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.01714717599999, 2023-10-30T06:26:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.792 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:13.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:13.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:13.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:13.805 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.806 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 109.01303071, 2023-10-30T06:26:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:13.807 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.808 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:13.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:13.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:13.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:13.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:13.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:13.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:13.821 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:13.822 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:13.514+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:13.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:13.824 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:13.821+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:13.924 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:13.922+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:13.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:13.990 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:13.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:14.025 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:14.024+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:14.125 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:14.126 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:14.126 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:14.124+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:14.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:14.227 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:14.226+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:14.304 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:14.304 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:14.304 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:14.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:14.328 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:14.326+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:14.429 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:14.427+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:14.439 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:14.439 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:14.440 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:14.440 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:14.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:14.530 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:14.529+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:14.574 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:14.574 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:14.574 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:14.575 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:14.575 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:14.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:14.631 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:14.629+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:14.717 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:14.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:14.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:14.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 38.426035663999997, 2023-10-30T06:26:14.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:14.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:14.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:14.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:14.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:14.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:14.721 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:14.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.721 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:14.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:14.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:14.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:14.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:14.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:14.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:14.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:14.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:14.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:14.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:14.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:14.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:14.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:14.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:14.731 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:14.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:14.732 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:14.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:14.736 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:14.733+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:14.836 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:14.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:14.873 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:14.873 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:14.874 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:14.874 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:14.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:14.936 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:14.935+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:15.037 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:15.036+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:15.052 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:15.052 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:15.052 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:15.053 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:15.053 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:15.062 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:15.146 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:15.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:15.227 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:15.229 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:15.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:15.279 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:15.276+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:15.341 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:15.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:15.405 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:15.406 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:15.406 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:15.406 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:15.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:15.458 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:15.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:15.541 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:15.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:15.541 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:15.542 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:15.542 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:15.542 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:15.540+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:15.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:15.643 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:15.641+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:15.744 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:15.742+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:15.760 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:15.760 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:15.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:15.760 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:15.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.761 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:15.761 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:15.761 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.602259124, 2023-10-30T06:26:15.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:15.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:15.762 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:15.762 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:15.762 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:15.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:15.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:15.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:15.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:15.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:15.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:15.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:15.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:15.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:15.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:15.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:15.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:15.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.772 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.773 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:15.774 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:15.774 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 120.689085261, 2023-10-30T06:26:15.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:15.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.775 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:15.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:15.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:15.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:15.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:15.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:15.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:15.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:15.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:15.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:15.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:15.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.785 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 120.028907292, 2023-10-30T06:26:15.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:15.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:15.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:15.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:15.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:15.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:15.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:15.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:15.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:15.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:15.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:15.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:15.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:15.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:15.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.797 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.953484085, 2023-10-30T06:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:15.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:15.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:15.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:15.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:15.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:15.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:15.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:15.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:15.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:15.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:15.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:15.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:15.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:15.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:15.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:15.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:15.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.812 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:15.813 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:15.813 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.682360954, 2023-10-30T06:26:15.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:15.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.814 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:15.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:15.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:15.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:15.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:15.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:15.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:15.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:15.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:15.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:15.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:15.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:15.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:15.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.823 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.824 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:15.824 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:15.825 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.652081244, 2023-10-30T06:26:15.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:15.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.825 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:15.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:15.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:15.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:15.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:15.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:15.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:15.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:15.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:15.834 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:15.835 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:15.836 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.27817016, 2023-10-30T06:26:15.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:15.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.836 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:15.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:15.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:15.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:15.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:15.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:15.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:15.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:15.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:15.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.847 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:15.847 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:15.847 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 119.273671459, 2023-10-30T06:26:15.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:15.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:15.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:15.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:15.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:15.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:15.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:15.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:15.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:15.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:15.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:15.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:15.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:15.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:15.861 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.98206752, 2023-10-30T06:26:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:15.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:15.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:15.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:15.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:15.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:15.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:15.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:15.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:15.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:15.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.871 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:15.872 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.976130203, 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:15.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:15.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:15.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:15.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:15.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:15.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:15.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:15.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:15.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:15.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:15.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:15.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:15.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:15.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:15.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:15.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:15.884 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.884 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.884 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.974831188, 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:15.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:15.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:15.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:15.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:15.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:15.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:15.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:15.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:15.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.895 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.895 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.896 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:15.896 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:15.896 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.87184238, 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:15.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:15.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:15.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:15.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:15.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:15.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:15.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:15.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:15.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.904 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:15.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:15.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:15.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:15.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.907 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.908 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:15.908 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:15.908 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.599939113, 2023-10-30T06:26:15.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:15.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.909 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:15.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:15.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:15.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:15.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:15.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:15.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:15.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:15.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:15.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:15.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:15.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:15.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:15.919 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.921 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:15.921 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:15.921 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.59244092100001, 2023-10-30T06:26:15.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:15.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:15.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:15.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:15.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:15.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:15.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:15.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:15.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:15.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:15.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:15.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:15.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.591779716, 2023-10-30T06:26:15.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:15.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:15.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:15.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:15.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:15.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:15.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:15.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:15.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:15.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:15.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:15.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:15.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:15.942 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.943 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.943 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.943 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:15.944 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:15.944 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.579193878, 2023-10-30T06:26:15.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:15.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.945 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:15.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:15.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:15.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:15.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:15.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:15.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:15.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:15.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:15.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:15.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:15.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:15.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:15.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:15.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:15.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:15.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:15.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:15.955 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.955 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.955 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.956 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:15.956 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:15.956 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.573101665, 2023-10-30T06:26:15.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:15.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.958 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:15.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:15.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:15.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:15.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:15.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:15.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:15.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:15.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:15.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:15.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:15.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:15.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:15.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:15.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:15.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.970 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.971 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:15.971 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.572106324, 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:15.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:15.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:15.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:15.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:15.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:15.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:15.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:15.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:15.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.981 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.982 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.982 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:15.982 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 111.24943375300001, 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:15.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:15.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:15.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:15.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:15.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:15.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:15.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:15.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:15.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:15.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:15.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:15.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:15.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:15.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:15.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:15.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.993 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:15.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:15.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.994 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:15.995 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:15.995 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 111.21599226799999, 2023-10-30T06:26:15.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:15.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:15.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:15.996 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:15.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:15.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:15.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:15.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:15.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:15.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:15.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:15.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:15.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:15.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:15.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:16.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:16.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:16.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:16.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:16.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:16.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:16.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:16.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:16.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:16.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:16.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:16.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:16.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:16.007 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:16.011 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:16.008+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:16.111 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:16.109+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:16.177 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:16.212 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:16.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:16.312 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:16.310+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:16.352 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:16.353 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:16.413 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:16.412+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:16.514 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:16.512+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:16.528 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.528 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:16.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:16.528 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:16.528 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:16.614 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:16.613+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:16.668 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.669 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:16.669 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:16.669 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:16.721 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:16.718+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:16.816 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:16.814+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:16.844 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.844 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:16.844 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:16.845 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:16.845 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:16.916 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:16.915+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:16.982 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:16.983 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:16.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:16.983 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:16.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.983 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:16.984 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:16.984 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 40.010498558000002, 2023-10-30T06:26:16.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:16.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:16.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:16.985 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:16.985 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:16.985 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:16.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:16.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.986 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:16.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:16.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:16.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:16.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:16.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:16.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:16.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:16.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:16.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:16.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:16.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:16.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:16.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:16.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:16.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:16.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:16.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:16.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:16.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:16.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:16.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:16.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:16.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.994 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:16.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:16.994 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:16.995 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:16.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:17.041 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:17.040+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:17.118 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:17.117+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:17.136 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:17.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:17.137 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:17.137 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:17.145 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:17.229 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:17.228+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:17.313 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:17.314 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:17.314 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:17.320 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:17.318+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:17.322 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:17.421 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:17.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:17.490 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:17.491 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:17.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:17.522 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:17.521+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:17.623 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:17.622+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:17.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:17.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:17.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:17.725 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:17.723+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:17.825 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:17.823+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:17.852 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:17.852 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:17.852 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:17.852 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:17.853 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:17.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:17.926 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:17.925+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:18.028 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:18.026+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:18.085 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:18.085 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:18.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:18.086 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:18.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.086 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:18.086 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:18.087 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 124.747857489, 2023-10-30T06:26:18.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:18.087 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.087 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.088 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:18.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.088 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:18.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:18.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:18.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:18.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:18.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:18.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:18.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:18.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:18.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:18.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:18.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:18.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:18.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:18.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:18.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:18.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:18.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:18.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:18.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:18.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:18.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:18.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.426702298, 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.106 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:18.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:18.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:18.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:18.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:18.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:18.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:18.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:18.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:18.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:18.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:18.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:18.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:18.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:18.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.120 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.422457741, 2023-10-30T06:26:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.122 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:18.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.122 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:18.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:18.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:18.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:18.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:18.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:18.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:18.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:18.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:18.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:18.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:18.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:18.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:18.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:18.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:18.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:18.135 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.136 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:18.137 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.336999316, 2023-10-30T06:26:18.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:18.137 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.137 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.137 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.138 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.138 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.138 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:18.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.138 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:18.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:18.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:18.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:18.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:18.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:18.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:18.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:18.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:18.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:18.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:18.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:18.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:18.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:18.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:18.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:18.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:18.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:18.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:18.151 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.151 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.152 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.153 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:18.153 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.0677561, 2023-10-30T06:26:18.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:18.153 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.153 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.154 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:18.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.155 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:18.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:18.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:18.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:18.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:18.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:18.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:18.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:18.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:18.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:18.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:18.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:18.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:18.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:18.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:18.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:18.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:18.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.168 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.066924689, 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:18.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.170 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.170 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:18.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:18.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:18.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:18.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:18.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:18.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:18.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:18.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:18.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:18.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:18.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:18.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:18.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:18.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.190 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.192 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.066257562, 2023-10-30T06:26:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:18.193 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:18.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:18.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:18.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:18.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:18.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:18.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:18.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.207 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.208 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.04398349100001, 2023-10-30T06:26:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:18.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:18.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:18.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:18.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:18.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.223 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.04333563500001, 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.225 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.04264956199999, 2023-10-30T06:26:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.240 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:18.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.971767594, 2023-10-30T06:26:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.97110362799999, 2023-10-30T06:26:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 120.942649819, 2023-10-30T06:26:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.600883326, 2023-10-30T06:26:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.60013465599999, 2023-10-30T06:26:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.599466273, 2023-10-30T06:26:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.589615599, 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.588941784, 2023-10-30T06:26:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.588293401, 2023-10-30T06:26:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 113.584176935, 2023-10-30T06:26:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:18.414 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:18.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:18.416 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:18.414+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:18.516 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:18.515+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:18.546 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:18.547 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:18.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:18.617 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:18.616+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:18.718 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:18.717+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:18.725 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:18.726 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:18.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:18.819 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:18.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:18.902 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:18.902 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:18.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:18.902 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:18.903 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:18.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:18.920 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:18.919+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:19.023 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:19.021+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:19.037 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:19.037 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:19.038 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:19.038 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:19.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:19.131 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:19.129+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:19.242 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:19.242 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:19.243 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:19.243 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:19.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:19.274 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:19.273+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:19.325 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:19.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:19.412 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:19.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:19.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 43.120978604000001, 2023-10-30T06:26:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:19.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:19.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:19.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:19.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:19.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:19.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:19.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:19.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:19.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:19.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:19.429 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:19.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:19.432 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:19.429+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:19.547 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:19.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:19.561 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:19.561 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:19.561 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:19.561 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:19.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:19.656 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:19.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:19.734 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:19.732+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:19.737 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:19.737 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:19.737 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:19.737 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:19.738 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:19.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:19.835 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:19.833+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:19.917 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:19.917 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:19.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:19.917 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:19.918 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:19.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:19.936 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:19.934+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:20.036 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:20.035+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:20.097 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:20.098 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:20.098 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:20.098 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:20.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:20.148 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:20.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:20.239 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:20.238+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:20.279 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:20.279 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:20.279 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:20.279 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:20.280 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:20.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:20.373 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:20.371+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:20.442 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:20.440+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:20.493 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:20.493 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:20.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:20.494 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:20.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.494 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:20.495 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:20.495 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.338664816, 2023-10-30T06:26:20.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:20.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.496 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:20.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:20.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:20.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:20.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:20.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:20.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:20.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:20.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:20.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:20.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:20.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:20.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 125.42549095299999, 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:20.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:20.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:20.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:20.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:20.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:20.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:20.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:20.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:20.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:20.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:20.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:20.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:20.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:20.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:20.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:20.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.518 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.519 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:20.519 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:20.519 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.765312984, 2023-10-30T06:26:20.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:20.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:20.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:20.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:20.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:20.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:20.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:20.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:20.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:20.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:20.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:20.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:20.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.530 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.531 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:20.531 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:20.531 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.689889777, 2023-10-30T06:26:20.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:20.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.532 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:20.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:20.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:20.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:20.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:20.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:20.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:20.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:20.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:20.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:20.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:20.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:20.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:20.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:20.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:20.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:20.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.542 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.543 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:20.543 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:20.543 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.41876664599999, 2023-10-30T06:26:20.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:20.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:20.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:20.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:20.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:20.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:20.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:20.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:20.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:20.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:20.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:20.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:20.554 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.38848693600001, 2023-10-30T06:26:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.555 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:20.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:20.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:20.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:20.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:20.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:20.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:20.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:20.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:20.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:20.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:20.566 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.01457585199999, 2023-10-30T06:26:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:20.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:20.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:20.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:20.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:20.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:20.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:20.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:20.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:20.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:20.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:20.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:20.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:20.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:20.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:20.577 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.577 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.577 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:20.578 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.010077151, 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:20.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:20.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:20.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:20.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:20.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:20.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:20.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:20.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:20.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:20.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:20.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.589 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.590 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:20.590 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:20.590 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.71847321200001, 2023-10-30T06:26:20.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:20.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.591 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:20.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:20.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:20.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:20.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:20.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:20.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:20.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:20.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:20.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:20.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:20.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:20.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:20.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.600 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.601 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.601 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:20.601 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:20.602 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.712535895, 2023-10-30T06:26:20.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:20.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.602 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:20.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:20.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:20.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:20.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:20.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:20.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:20.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:20.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:20.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:20.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:20.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:20.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:20.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:20.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:20.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.612 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.71123688, 2023-10-30T06:26:20.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:20.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.614 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:20.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:20.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:20.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:20.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:20.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:20.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:20.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:20.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:20.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:20.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:20.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:20.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.624 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.60824807199999, 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.626 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:20.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:20.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:20.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:20.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:20.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:20.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:20.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:20.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:20.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:20.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:20.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:20.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:20.635 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.635 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:20.636 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.336344805, 2023-10-30T06:26:20.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:20.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.637 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:20.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:20.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:20.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:20.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:20.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:20.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:20.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:20.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:20.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:20.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:20.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:20.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:20.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.646 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.648 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:20.648 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:20.648 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.328846613, 2023-10-30T06:26:20.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:20.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.649 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:20.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:20.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:20.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:20.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:20.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:20.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:20.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:20.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:20.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:20.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:20.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:20.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:20.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:20.659 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:20.660 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.328185408, 2023-10-30T06:26:20.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:20.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.660 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:20.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:20.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:20.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:20.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:20.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:20.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:20.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:20.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:20.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:20.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:20.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:20.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:20.670 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.670 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.670 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.671 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:20.671 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:20.671 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.31559957, 2023-10-30T06:26:20.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:20.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.672 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:20.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:20.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:20.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:20.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:20.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.682 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:20.683 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.309507357, 2023-10-30T06:26:20.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:20.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.684 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:20.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:20.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:20.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:20.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:20.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.695 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:20.695 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:20.695 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 123.30851201599999, 2023-10-30T06:26:20.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:20.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.696 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:20.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:20.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:20.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:20.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:20.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:20.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:20.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:20.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:20.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:20.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.706 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:20.706 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:20.707 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.985839445, 2023-10-30T06:26:20.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:20.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.707 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:20.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:20.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:20.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:20.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:20.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:20.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:20.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:20.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:20.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:20.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:20.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:20.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.718 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:20.718 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:20.718 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 115.95239796, 2023-10-30T06:26:20.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:20.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:20.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:20.719 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:20.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:20.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:20.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:20.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:20.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:20.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:20.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:20.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:20.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:20.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:20.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:20.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:20.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:20.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:20.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:20.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:20.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:20.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:20.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:20.728 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:20.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:20.729 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:20.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:20.732 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:20.730+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:20.832 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:20.831+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:20.899 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:20.900 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:20.900 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:20.900 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:20.908 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:20.933 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:20.932+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:21.034 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:21.033+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:21.076 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:21.076 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:21.076 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:21.077 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:21.077 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:21.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:21.136 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:21.134+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:21.237 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:21.235+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:21.251 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:21.252 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:21.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:21.338 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:21.337+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:21.388 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:21.388 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:21.389 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:21.389 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:21.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:21.441 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:21.438+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:21.540 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:21.538+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:21.565 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:21.565 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:21.565 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:21.565 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:21.566 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:21.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:21.641 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:21.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:21.705 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:21.706 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:21.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:21.706 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:21.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.707 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:21.707 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:21.707 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 44.733012967000001, 2023-10-30T06:26:21.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:21.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:21.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:21.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:21.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:21.708 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:21.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:21.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:21.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:21.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:21.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:21.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:21.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:21.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:21.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:21.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:21.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:21.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:21.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:21.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:21.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:21.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:21.716 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:21.717 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:21.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:21.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:21.717 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:21.717 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:21.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:21.763 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:21.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:21.843 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:21.841+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:21.856 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:21.856 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:21.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:21.944 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:21.943+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:22.038 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:22.039 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:22.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:22.046 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:22.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:22.158 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:22.156+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:22.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:22.172 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:22.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:22.247 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:22.246+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:22.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:22.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:22.349 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:22.347+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:22.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:22.448 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:22.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:22.481 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:22.481 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:22.482 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:22.482 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:22.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:22.550 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:22.548+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:22.651 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:22.650+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:22.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:22.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:22.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 129.33191812499999, 2023-10-30T06:26:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.675 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:22.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:22.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:22.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:22.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:22.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:22.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.690 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:22.691 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 127.010762934, 2023-10-30T06:26:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.692 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 127.00651837700001, 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:22.707 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.708 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.92105995199999, 2023-10-30T06:26:22.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.724 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:22.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:22.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:22.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:22.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:22.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:22.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:22.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:22.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:22.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:22.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:22.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:22.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:22.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:22.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:22.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:22.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:22.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:22.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:22.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:22.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.651816736, 2023-10-30T06:26:22.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:22.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:22.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:22.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:22.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:22.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:22.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:22.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:22.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:22.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:22.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:22.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.65098532499999, 2023-10-30T06:26:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:22.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:22.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.65031819799999, 2023-10-30T06:26:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.628044127, 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.62739627099999, 2023-10-30T06:26:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 126.626710198, 2023-10-30T06:26:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.55582823, 2023-10-30T06:26:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.555164264, 2023-10-30T06:26:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 125.526710455, 2023-10-30T06:26:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.18494396200001, 2023-10-30T06:26:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.184195292, 2023-10-30T06:26:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.18352690899999, 2023-10-30T06:26:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.173676235, 2023-10-30T06:26:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.17300242, 2023-10-30T06:26:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.17235403700001, 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.16823757100001, 2023-10-30T06:26:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:22.992 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:22.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:22.994 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:22.992+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:23.095 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:23.094+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:23.123 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:23.123 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:23.124 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:23.196 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:23.194+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:23.258 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.258 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:23.259 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:23.259 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:23.259 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:23.296 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:23.295+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:23.395 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:23.396 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.398 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:23.396+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:23.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:23.499 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:23.497+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:23.550 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:23.551 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:23.600 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:23.598+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:23.683 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:23.701 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:23.700+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:23.802 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:23.801+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:23.863 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:23.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:23.864 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 47.571161658000001, 2023-10-30T06:26:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:23.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:23.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:23.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:23.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:23.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:23.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:23.874 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:23.876 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:23.921 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:23.919+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:24.004 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:24.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:24.014 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:24.014 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:24.014 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:24.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:24.105 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:24.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:24.195 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.195 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:24.196 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:24.196 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:24.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:24.206 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:24.205+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:24.322 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:24.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:24.346 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:24.347 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:24.347 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:24.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:24.439 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:24.437+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:24.509 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:24.507+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:24.522 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:24.522 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:24.522 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:24.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:24.617 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:24.615+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:24.698 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.698 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:24.699 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:24.699 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:24.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:24.711 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:24.709+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:24.813 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:24.811+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:24.914 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:24.912+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:24.916 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:24.916 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.75876838600001, 2023-10-30T06:26:24.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:24.918 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:24.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:24.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:24.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 129.84559452299999, 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 129.185416554, 2023-10-30T06:26:24.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:24.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:24.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:24.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 129.109993347, 2023-10-30T06:26:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:24.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:24.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:24.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:24.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:24.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:24.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:24.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:24.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:24.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:24.969 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.838870216, 2023-10-30T06:26:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.970 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:24.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:24.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:24.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.980 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:24.981 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.808590506, 2023-10-30T06:26:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:24.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:24.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:24.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:24.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.992 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:24.993 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.43467942199999, 2023-10-30T06:26:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:24.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.430180721, 2023-10-30T06:26:25.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:25.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:25.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:25.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:25.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:25.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:25.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:25.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:25.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.017 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:25.018 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.138576782, 2023-10-30T06:26:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.019 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:25.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.029 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:25.030 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.13263946500001, 2023-10-30T06:26:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.13134045000001, 2023-10-30T06:26:25.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:25.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:25.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:25.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.02835164199999, 2023-10-30T06:26:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:25.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:25.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:25.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:25.065 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.75644837500001, 2023-10-30T06:26:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:25.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:25.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.74895018300001, 2023-10-30T06:26:25.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:25.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:25.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:25.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:25.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.088 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.74828897800001, 2023-10-30T06:26:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.089 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:25.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:25.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:25.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:25.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:25.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:25.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.099 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:25.100 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.73570314, 2023-10-30T06:26:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.101 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.729610927, 2023-10-30T06:26:25.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:25.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:25.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 127.728615586, 2023-10-30T06:26:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:25.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:25.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:25.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:25.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:25.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:25.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:25.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.134 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:25.135 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 120.40594301500001, 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 120.37250152999999, 2023-10-30T06:26:25.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:25.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:25.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:25.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:25.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:25.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:25.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:25.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:25.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:25.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:25.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:25.158 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:25.160 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:25.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:25.261 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:25.259+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:25.327 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:25.327 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:25.327 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:25.362 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:25.360+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:25.463 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:25.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:25.503 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:25.503 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:25.504 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:25.564 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:25.562+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:25.639 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:25.640 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:25.694 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:25.692+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:25.766 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:25.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:25.780 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:25.781 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:25.875 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:25.873+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:25.955 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:25.956 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:25.956 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:25.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:26.005 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:26.004+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:26.068 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:26.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:26.132 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 49.160456672000002, 2023-10-30T06:26:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:26.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:26.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:26.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:26.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:26.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:26.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:26.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:26.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:26.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:26.145 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:26.145 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:26.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:26.191 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:26.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:26.270 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:26.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:26.284 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:26.285 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:26.285 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:26.285 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:26.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:26.372 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:26.370+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:26.463 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:26.464 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:26.464 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:26.464 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:26.464 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:26.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:26.472 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:26.471+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:26.573 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:26.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:26.597 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:26.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:26.598 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:26.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:26.675 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:26.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:26.774 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:26.775 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:26.775 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:26.776 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:26.773+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:26.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:26.876 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:26.874+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:26.950 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:26.950 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:26.951 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:26.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:27.005 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:27.003+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:27.077 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:27.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:27.140 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:27.140 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:27.141 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 133.80383017599999, 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:27.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:27.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:27.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:27.157 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.158 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.48267498499999, 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.159 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.160 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:27.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.478430428, 2023-10-30T06:26:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:27.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.175 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:27.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:27.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:27.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:27.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:27.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:27.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.39297200300001, 2023-10-30T06:26:27.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:27.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:27.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:27.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:27.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:27.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.123728787, 2023-10-30T06:26:27.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.205 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:27.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:27.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:27.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:27.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:27.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.218 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.122897376, 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:27.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:27.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.12223024900001, 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.09995617800001, 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.09930832200001, 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 131.09862224899999, 2023-10-30T06:26:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.02774028100001, 2023-10-30T06:26:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.02707631499999, 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 129.998622506, 2023-10-30T06:26:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.656856013, 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.656107343, 2023-10-30T06:26:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.65543896, 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.64558828600001, 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.64491447100001, 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.64426608799999, 2023-10-30T06:26:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 122.640149622, 2023-10-30T06:26:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:27.446 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:27.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:27.449 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:27.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:27.557 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:27.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:27.619 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:27.619 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:27.619 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:27.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:27.649 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:27.648+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:27.750 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:27.749+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:27.818 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:27.818 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:27.819 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:27.819 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:27.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:27.873 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:27.871+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:27.951 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:27.950+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:27.960 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:27.960 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:27.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:27.960 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:27.961 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:27.969 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:28.053 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:28.051+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:28.135 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:28.135 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:28.135 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:28.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:28.185 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:28.183+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:28.253 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:28.252+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:28.308 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:28.309 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:28.309 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:28.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:28.362 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:28.360+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:28.447 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:28.447 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:28.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 52.155545660000001, 2023-10-30T06:26:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:28.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:28.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:28.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:28.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:28.459 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:28.459 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:28.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:28.461 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:28.460+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:28.561 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:28.560+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:28.599 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:28.600 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:28.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:28.698 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:28.696+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:28.763 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:28.762+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:28.779 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:28.780 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:28.780 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:28.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:28.881 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:28.877+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:28.959 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:28.960 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:28.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:28.960 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:28.960 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:28.965 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:28.964+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:28.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:29.070 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:29.068+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:29.137 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:29.137 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:29.137 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:29.138 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:29.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:29.195 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:29.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:29.269 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:29.268+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:29.275 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:29.276 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:29.276 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:29.276 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:29.276 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:29.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:29.370 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:29.369+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:29.494 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:29.495 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:29.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:29.495 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:29.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.496 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:29.496 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:29.496 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.33874788099999, 2023-10-30T06:26:29.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:29.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:29.497 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:29.497 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:29.497 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:29.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:29.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.498 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:29.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:29.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:29.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:29.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:29.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:29.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:29.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:29.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:29.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:29.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:29.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:29.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:29.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:29.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:29.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:29.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:29.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:29.511 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.511 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.512 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.512 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:29.512 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:29.513 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 134.42557401799999, 2023-10-30T06:26:29.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:29.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.513 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:29.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:29.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:29.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:29.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:29.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:29.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:29.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:29.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:29.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:29.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:29.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:29.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:29.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.523 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.524 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.524 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:29.524 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:29.525 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.765396049, 2023-10-30T06:26:29.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:29.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.526 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:29.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:29.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:29.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:29.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:29.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:29.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:29.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:29.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:29.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:29.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:29.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:29.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:29.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.535 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.536 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.536 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:29.537 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:29.537 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.689972842, 2023-10-30T06:26:29.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:29.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.538 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:29.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:29.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:29.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:29.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:29.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:29.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:29.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:29.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:29.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:29.548 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.549 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.41884971100001, 2023-10-30T06:26:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:29.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:29.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:29.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:29.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:29.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:29.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:29.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:29.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:29.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:29.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:29.560 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.560 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.561 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.561 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:29.561 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.38857000100001, 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:29.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:29.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:29.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:29.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:29.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:29.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:29.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:29.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:29.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:29.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:29.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:29.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.573 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:29.573 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:29.573 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.01465891699999, 2023-10-30T06:26:29.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:29.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.574 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:29.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:29.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:29.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:29.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:29.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:29.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:29.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:29.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:29.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:29.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:29.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:29.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:29.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:29.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:29.583 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.584 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.584 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:29.585 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:29.585 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 133.010160216, 2023-10-30T06:26:29.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:29.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.586 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:29.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:29.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:29.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:29.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:29.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:29.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:29.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:29.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:29.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:29.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:29.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:29.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:29.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.596 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:29.596 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:29.596 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.718556277, 2023-10-30T06:26:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:29.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:29.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:29.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:29.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:29.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:29.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:29.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:29.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:29.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:29.607 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.607 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.607 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.608 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:29.608 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:29.608 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.71261895999999, 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:29.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:29.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:29.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:29.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:29.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:29.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:29.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:29.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:29.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:29.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:29.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:29.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:29.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:29.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:29.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:29.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.619 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.620 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:29.620 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:29.620 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.71131994500001, 2023-10-30T06:26:29.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:29.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.621 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:29.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:29.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:29.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:29.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:29.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:29.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:29.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:29.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:29.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:29.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:29.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:29.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:29.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:29.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.60833113699999, 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.633 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:29.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:29.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:29.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:29.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:29.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:29.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:29.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:29.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:29.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:29.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:29.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:29.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:29.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:29.642 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.642 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.643 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.643 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:29.643 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.33642786999999, 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:29.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.646 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:29.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:29.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:29.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:29.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:29.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:29.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:29.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:29.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:29.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:29.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:29.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:29.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:29.654 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.654 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.654 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.32892967800001, 2023-10-30T06:26:29.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:29.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.656 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:29.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:29.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:29.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:29.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:29.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:29.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:29.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:29.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:29.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:29.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:29.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:29.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:29.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.666 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:29.667 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:29.667 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.32826847300001, 2023-10-30T06:26:29.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:29.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.668 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:29.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:29.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:29.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:29.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:29.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:29.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:29.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:29.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:29.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:29.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:29.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:29.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:29.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:29.677 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.678 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.678 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.315682635, 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.679 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:29.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:29.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:29.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:29.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:29.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:29.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:29.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:29.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:29.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:29.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:29.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:29.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:29.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:29.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.689 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.690 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:29.690 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:29.690 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.30959042200001, 2023-10-30T06:26:29.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:29.691 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.691 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.691 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:29.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:29.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:29.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:29.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:29.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:29.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:29.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:29.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:29.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:29.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:29.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:29.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.702 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:29.702 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:29.702 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 132.30859508099999, 2023-10-30T06:26:29.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:29.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.703 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:29.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:29.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:29.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:29.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:29.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:29.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:29.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:29.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:29.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:29.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:29.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:29.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:29.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:29.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.714 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.715 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:29.715 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:29.715 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.98592250999999, 2023-10-30T06:26:29.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:29.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.716 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:29.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:29.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:29.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:29.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:29.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:29.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:29.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:29.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:29.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:29.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:29.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:29.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.727 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:29.727 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:29.727 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.952481025, 2023-10-30T06:26:29.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:29.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:29.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:29.728 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:29.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:29.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:29.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:29.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:29.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:29.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:29.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:29.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:29.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:29.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:29.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:29.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:29.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:29.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:29.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:29.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:29.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:29.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:29.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:29.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:29.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:29.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:29.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:29.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:29.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:29.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:29.738 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.738 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.738 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.738 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:29.738 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:29.739 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:29.739 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:29.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:29.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:29.741 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:26:29.419+0000 d76a640 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is log(1 entries from seq 13 at 2023-10-30T06:25:57.757046+0000) 2023-10-30T06:26:29.742 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:29.742 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 44, 2023-10-30T06:26:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:26:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:26:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:25:55.691575+0000", 2023-10-30T06:26:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:26:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:26:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:26:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2023-10-30T06:26:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:26:29.744 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2023-10-30T06:26:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2023-10-30T06:26:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:26:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:26:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:26:29.745 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:26:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:26:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:26:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:26:29.746 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:26:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:26:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:26:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:26:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:26:29.747 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:26:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:26:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:26:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:26:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.748 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:26:29.749 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:26:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:26:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:26:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:26:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:26:29.750 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:26:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:26:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:26:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:26:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:26:29.751 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:26:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:26:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:26:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:26:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:26:29.752 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:26:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:26:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:26:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:26:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:26:29.753 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:26:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:26:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:26:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:26:29.754 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:26:29.755 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:26:29.756 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:26:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:26:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:26:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:26:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:26:29.757 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:26:29.758 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:26:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:26:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:26:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:26:29.759 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:26:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:26:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:26:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:26:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:26:29.760 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:26:29.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:26:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:26:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:26:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.762 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:26:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:26:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:26:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:26:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:26:29.763 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:26:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:26:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:26:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:26:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:26:29.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:26:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:26:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:26:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:26:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:26:29.766 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:26:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:26:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:26:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:26:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:26:29.767 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:26:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:26:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:26:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:26:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:26:29.768 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:26:29.769 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:26:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:26:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:26:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:26:29.770 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:26:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:26:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:26:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:26:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:26:29.771 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:26:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:26:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:26:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:26:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:26:29.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:26:29.773 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:26:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:26:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:26:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:26:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:26:29.774 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:26:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:26:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:26:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:26:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.775 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:26:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:26:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:26:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:26:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:26:29.776 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:26:29.777 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:26:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:26:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:26:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:26:29.778 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.630000114440918, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.630000114440918, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:26:29.779 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:26:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:26:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:26:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:26:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:26:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.781 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.782 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:26:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:26:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:26:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:26:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:26:29.789 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.790 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.790 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:26:29.790 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.791 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.791 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.791 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:26:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:26:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:29.793 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:26:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:26:29.794 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:26:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:26:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.796 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.797 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:26:29.798 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 33, 2023-10-30T06:26:29.798 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.798 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.798 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.798 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.799 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.799 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:26:29.799 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.800 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:26:29.800 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.801 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.801 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.803 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:26:29.803 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.804 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.804 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.804 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:26:29.805 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.806 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.806 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.807 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.807 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:26:29.807 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.808 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.809 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:26:29.809 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:26:29.809 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:26:29.809 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:26:29.810 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.810 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.810 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.810 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.811 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:26:29.811 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:26:29.811 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.811 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:29.812 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 26, 2023-10-30T06:26:29.813 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.813 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.813 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.813 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.814 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:26:29.814 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.816 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:26:29.817 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.817 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:26:29.818 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.818 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.820 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:26:29.820 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.821 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.821 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.821 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.822 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.822 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:26:29.822 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.824 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:26:29.824 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:26:29.824 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:26:29.824 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:26:29.824 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.825 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.825 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.826 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:26:29.826 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:26:29.826 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.826 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.826 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.827 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.827 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.827 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.827 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:29.827 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.829 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:26:29.829 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.830 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:26:29.830 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:26:29.831 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.832 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:26:29.832 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.834 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:26:29.834 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.835 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:26:29.835 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.836 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:26:29.837 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.837 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.837 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:26:29.838 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:29.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.838 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:26:29.839 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:26:29.839 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:26:29.839 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:26:29.839 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.839 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:26:29.840 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:26:29.841 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.841 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.841 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.841 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.841 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.842 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.842 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:29.842 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:29.842 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.842 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.843 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.843 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.843 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:26:29.844 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.844 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.845 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.846 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:26:29.846 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:26:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.848 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.848 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.849 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:26:29.849 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.850 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:26:29.850 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.851 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:26:29.852 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.852 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.852 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:26:29.853 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:29.853 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.853 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:26:29.854 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:26:29.854 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:26:29.854 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:26:29.854 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.854 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:26:29.855 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:26:29.856 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.856 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.856 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.856 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.856 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.857 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.858 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.858 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:26:29.858 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.859 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:26:29.859 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.861 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:26:29.861 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.862 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:26:29.862 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.863 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:26:29.864 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.864 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.865 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.866 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:26:29.866 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.867 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.867 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:26:29.867 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:29.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:26:29.868 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:26:29.869 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.869 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.869 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:26:29.870 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:26:29.870 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.870 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:29.871 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:26:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:26:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.875 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:26:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.876 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.877 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:26:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:26:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.882 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:26:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:26:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:26:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:26:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.883 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.884 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:29.885 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:26:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:29.886 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.887 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:26:29.888 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:29.889 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:26:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:26:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.894 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:26:29.895 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:26:29.896 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:26:29.897 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:26:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:26:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:29.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:26:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.899 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:26:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.900 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:26:29.901 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.902 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.903 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:26:29.904 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.905 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:26:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:26:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.907 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:26:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:26:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.909 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:26:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.911 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:26:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:26:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:29.913 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:26:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.914 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:26:29.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:26:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:26:29.917 INFO:teuthology.orchestra.run.smithi079.stdout: 2, 2023-10-30T06:26:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: 1 2023-10-30T06:26:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.918 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:26:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.3", 2023-10-30T06:26:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:26:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:29.919 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 7 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:29.920 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:26:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:26:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:26:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:26:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:26:29.921 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:26:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:26:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:26:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:26:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:26:29.922 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:26:29.923 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:26:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:26:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:26:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:26:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:26:29.924 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:26:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:26:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:26:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:26:29.925 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:29.925 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:29.926 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:25:55.691575+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [2, 1]}], 'pg_upmap_items': [{'pgid': '2.3', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:26:29.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:26:29.927 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:29.928 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:29.929 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:29.929 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:26:29.930 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T06:26:29.930 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-30T06:26:29.931 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:29.740+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:29.931 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:29.931 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:29.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:29.933 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:29.934 INFO:tasks.ceph.mon.b:Started 2023-10-30T06:26:29.934 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:26:29.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:29.972 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:29.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:29.973 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:26:29.983 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:29.983 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:30.034 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:30.032+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:30.063 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:30.063 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:30.063 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:30.064 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:30.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:30.096 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:30.096 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:26:30.158 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:30.156+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:30.184 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:30.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:26:30.236 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:30.234+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:30.276 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:30.277 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:26:30.278 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:30.278 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:30.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:30.337 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:30.335+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:30.368 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:26:30.438 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:30.436+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:30.458 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:30.459 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:30.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:30.539 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:30.538+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:30.598 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:30.598 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:30.599 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:30.600 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:26:30.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2023-10-30T06:26:29.912380+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220040,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754680,"statfs":{"total":773094113280,"available":772868792320,"internally_reserved":0,"allocated":3973120,"data_stored":2336040,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.633150"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.132621+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:25:52.132621+0000","last_peered":"2023-10-30T06:25:52.132621+0000","last_clean":"2023-10-30T06:25:52.132621+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:25:52.132621+0000","last_undegraded":"2023-10-30T06:25:52.132621+0000","last_fullsized":"2023-10-30T06:25:52.132621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:53.110947+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:25:53.110947+0000","last_peered":"2023-10-30T06:25:53.110947+0000","last_clean":"2023-10-30T06:25:53.110947+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:25:53.110947+0000","last_undegraded":"2023-10-30T06:25:53.110947+0000","last_fullsized":"2023-10-30T06:25:53.110947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.203651+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:25:52.203651+0000","last_peered":"2023-10-30T06:25:52.203651+0000","last_clean":"2023-10-30T06:25:52.203651+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:25:52.203651+0000","last_undegraded":"2023-10-30T06:25:52.203651+0000","last_fullsized":"2023-10-30T06:25:52.203651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-30T06:25:56.171645+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:25:56.171645+0000","last_peered":"2023-10-30T06:25:56.171645+0000","last_clean":"2023-10-30T06:25:56.171645+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:25:56.171645+0000","last_undegraded":"2023-10-30T06:25:56.171645+0000","last_fullsized":"2023-10-30T06:25:56.171645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-30T06:25:56.172183+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:25:56.172183+0000","last_peered":"2023-10-30T06:25:56.172183+0000","last_clean":"2023-10-30T06:25:56.172183+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:25:56.172183+0000","last_undegraded":"2023-10-30T06:25:56.172183+0000","last_fullsized":"2023-10-30T06:25:56.172183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-30T06:25:55.891419+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:25:55.891419+0000","last_peered":"2023-10-30T06:25:55.891419+0000","last_clean":"2023-10-30T06:25:55.891419+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:25:55.891419+0000","last_undegraded":"2023-10-30T06:25:55.891419+0000","last_fullsized":"2023-10-30T06:25:55.891419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-30T06:25:56.174058+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:25:56.174058+0000","last_peered":"2023-10-30T06:25:56.174058+0000","last_clean":"2023-10-30T06:25:56.174058+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:25:56.174058+0000","last_undegraded":"2023-10-30T06:25:56.174058+0000","last_fullsized":"2023-10-30T06:25:56.174058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-30T06:25:52.690771+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:25:52.690771+0000","last_peered":"2023-10-30T06:25:52.690771+0000","last_clean":"2023-10-30T06:25:52.690771+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:25:52.690771+0000","last_undegraded":"2023-10-30T06:25:52.690771+0000","last_fullsized":"2023-10-30T06:25:52.690771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":138,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-30T06:25:56.160876+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:25:56.160876+0000","last_peered":"2023-10-30T06:25:56.160876+0000","last_clean":"2023-10-30T06:25:56.160876+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:25:56.160876+0000","last_undegraded":"2023-10-30T06:25:56.160876+0000","last_fullsized":"2023-10-30T06:25:56.160876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":590385,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":11.029999999999999,"5min":10.494,"15min":10.404},"min":{"1min":5.2720000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"last":5.4699999999999998},{"interface":"front","average":{"1min":12.122999999999999,"5min":12.090999999999999,"15min":12.085000000000001},"min":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":35.398000000000003,"5min":46.798000000000002,"15min":46.798000000000002},"last":9.9949999999999992}]},{"osd":2,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":13.641,"15min":13.744999999999999},"min":{"1min":5.7350000000000003,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":44.537999999999997,"5min":52.552,"15min":52.552},"last":6.9669999999999996},{"interface":"front","average":{"1min":17.143000000000001,"5min":15.545999999999999,"15min":15.279999999999999},"min":{"1min":7.391,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":55.585999999999999,"5min":55.585999999999999,"15min":55.585999999999999},"last":14.018000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":15.367000000000001,"5min":21.626000000000001,"15min":22.669},"min":{"1min":8.5980000000000008,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":32.259999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":13.193},{"interface":"front","average":{"1min":14.832000000000001,"5min":21.024000000000001,"15min":22.056000000000001},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":38.987000000000002,"5min":124.075,"15min":124.075},"last":8.7569999999999997}]},{"osd":4,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":13.747,"15min":13.699},"min":{"1min":7.6470000000000002,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":25.023,"5min":51.649000000000001,"15min":51.649000000000001},"last":9.2759999999999998},{"interface":"front","average":{"1min":18.245000000000001,"5min":18.047999999999998,"15min":18.015000000000001},"min":{"1min":10.759,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":44.973999999999997,"5min":44.973999999999997,"15min":44.973999999999997},"last":16.399999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":16.564,"5min":20.154,"15min":20.751999999999999},"min":{"1min":7.351,"5min":7.351,"15min":7.351},"max":{"1min":58.648000000000003,"5min":123.754,"15min":123.754},"last":10.718999999999999},{"interface":"front","average":{"1min":14.667999999999999,"5min":21.396999999999998,"15min":22.518999999999998},"min":{"1min":7.6619999999999999,"5min":7.6619999999999999,"15min":7.6619999999999999},"max":{"1min":41.558999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":14.718999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":17.972000000000001,"5min":20.760000000000002,"15min":21.224},"min":{"1min":9.359,"5min":9.359,"15min":9.359},"max":{"1min":39.860999999999997,"5min":66.400999999999996,"15min":66.400999999999996},"last":11.858000000000001},{"interface":"front","average":{"1min":16.838999999999999,"5min":19.254999999999999,"15min":19.657},"min":{"1min":9.6969999999999992,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":37.027999999999999,"5min":53.149000000000001,"15min":53.149000000000001},"last":15.795}]},{"osd":7,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":18.468,"5min":20.904,"15min":21.309999999999999},"min":{"1min":10.083,"5min":10.083,"15min":10.083},"max":{"1min":51.375,"5min":51.375,"15min":51.375},"last":17.062000000000001},{"interface":"front","average":{"1min":19.795999999999999,"5min":21.234999999999999,"15min":21.475000000000001},"min":{"1min":10.004,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":54.439,"5min":69.991,"15min":69.991},"last":11.282}]}]},{"osd":6,"up_from":16,"seq":68719476768,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.561,"5min":13.877000000000001,"15min":14.429},"min":{"1min":4.9980000000000002,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":37.701999999999998,"5min":94.343000000000004,"15min":94.343000000000004},"last":9.1280000000000001},{"interface":"front","average":{"1min":11.94,"5min":14.991,"15min":15.499000000000001},"min":{"1min":5.8700000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":49.045999999999999,"5min":122.054,"15min":122.054},"last":6.8010000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.071,"5min":10.667,"15min":10.766},"min":{"1min":5.8789999999999996,"5min":5.8789999999999996,"15min":5.8789999999999996},"max":{"1min":16.489999999999998,"5min":18.992999999999999,"15min":18.992999999999999},"last":7.5250000000000004},{"interface":"front","average":{"1min":13.263,"5min":29.742999999999999,"15min":32.490000000000002},"min":{"1min":8.984,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":19.466999999999999,"5min":387.15600000000001,"15min":387.15600000000001},"last":10.927}]},{"osd":2,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.083,"5min":18.381,"15min":19.431000000000001},"min":{"1min":6.5800000000000001,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":31.495999999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":12.282},{"interface":"front","average":{"1min":14.241,"5min":19.315000000000001,"15min":20.161000000000001},"min":{"1min":7.7249999999999996,"5min":7.7249999999999996,"15min":7.7249999999999996},"max":{"1min":20.09,"5min":143.578,"15min":143.578},"last":14.641999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.124000000000001,"5min":20.594999999999999,"15min":21.507000000000001},"min":{"1min":9.6370000000000005,"5min":9.6370000000000005,"15min":9.6370000000000005},"max":{"1min":21.492999999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":13.573},{"interface":"front","average":{"1min":18.352,"5min":24.007000000000001,"15min":24.949000000000002},"min":{"1min":13.48,"5min":13.48,"15min":13.48},"max":{"1min":30.067,"5min":161.32900000000001,"15min":161.32900000000001},"last":15.818}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":11.082000000000001,"5min":14.252000000000001,"15min":14.779999999999999},"min":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":18.454000000000001,"5min":74.079999999999998,"15min":74.079999999999998},"last":13.005000000000001},{"interface":"front","average":{"1min":13.582000000000001,"5min":15.278,"15min":15.56},"min":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"max":{"1min":20.756,"5min":58.613999999999997,"15min":58.613999999999997},"last":9.7989999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":13.81,"5min":17.134,"15min":17.687999999999999},"min":{"1min":7.383,"5min":7.383,"15min":7.383},"max":{"1min":20.297000000000001,"5min":71.757000000000005,"15min":71.757000000000005},"last":17.940000000000001},{"interface":"front","average":{"1min":17.872,"5min":20.984000000000002,"15min":21.501999999999999},"min":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"max":{"1min":24.132000000000001,"5min":80.119,"15min":80.119},"last":15.247}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.315000000000001,"5min":19.055,"15min":19.512},"min":{"1min":8.9610000000000003,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":29.869,"5min":81.247,"15min":81.247},"last":17.138999999999999},{"interface":"front","average":{"1min":16.158000000000001,"5min":18.501999999999999,"15min":18.891999999999999},"min":{"1min":10.763,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":27.731999999999999,"5min":70.989000000000004,"15min":70.989000000000004},"last":10.369999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476768,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":10.17,"5min":10.882,"15min":11},"min":{"1min":6.6020000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":26.021000000000001,"5min":26.021000000000001,"15min":26.021000000000001},"last":7.1459999999999999},{"interface":"front","average":{"1min":11.24,"5min":10.454000000000001,"15min":10.323},"min":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":27.683,"5min":27.683,"15min":27.683},"last":9.1880000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":12.688000000000001,"5min":23.998999999999999,"15min":25.884},"min":{"1min":7.4039999999999999,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":22.036000000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":8.4369999999999994},{"interface":"front","average":{"1min":12.916,"5min":25.045999999999999,"15min":27.068000000000001},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24.646999999999998,"5min":190.34200000000001,"15min":190.34200000000001},"last":10.928000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.867000000000001,"5min":17.914999999999999,"15min":18.257000000000001},"min":{"1min":10.250999999999999,"5min":10.250999999999999,"15min":10.250999999999999},"max":{"1min":30.335000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":14.817},{"interface":"front","average":{"1min":14.234,"5min":16.606000000000002,"15min":17.001000000000001},"min":{"1min":9.1639999999999997,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":23.334,"5min":33.060000000000002,"15min":33.060000000000002},"last":15.432}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.425000000000001,"5min":19.581,"15min":20.273},"min":{"1min":10.442,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":23.326000000000001,"5min":42.274999999999999,"15min":42.274999999999999},"last":18.710000000000001},{"interface":"front","average":{"1min":9.7880000000000003,"5min":11.247999999999999,"15min":11.492000000000001},"min":{"1min":5.7969999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":19.716000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":11.923}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":18.329999999999998,"15min":19.216000000000001},"min":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"max":{"1min":22.614000000000001,"5min":56.796999999999997,"15min":56.796999999999997},"last":10.065},{"interface":"front","average":{"1min":15.929,"5min":19.824999999999999,"15min":20.474},"min":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"max":{"1min":28.501000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":17.456}]},{"osd":6,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":19.527999999999999,"5min":27.216000000000001,"15min":28.497},"min":{"1min":10.875,"5min":10.875,"15min":10.875},"max":{"1min":41.771000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":19.552},{"interface":"front","average":{"1min":14.746,"5min":15.343999999999999,"15min":15.444000000000001},"min":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"max":{"1min":40.128999999999998,"5min":40.128999999999998,"15min":40.128999999999998},"last":16.826000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":16.882999999999999,"5min":17.765000000000001,"15min":17.911999999999999},"min":{"1min":10.404,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":31.125,"5min":38.887999999999998,"15min":38.887999999999998},"last":28.684999999999999},{"interface":"front","average":{"1min":19.367999999999999,"5min":27.565000000000001,"15min":28.931000000000001},"min":{"1min":12.272,"5min":10.025,"15min":10.025},"max":{"1min":29.981999999999999,"5min":170.232,"15min":170.232},"last":26.358000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476766,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.563000000000001,"5min":24.759,"15min":26.792000000000002},"min":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"max":{"1min":51.853000000000002,"5min":258.66199999999998,"15min":258.66199999999998},"last":7.5350000000000001},{"interface":"front","average":{"1min":12.401,"5min":21.513999999999999,"15min":23.033000000000001},"min":{"1min":6.6879999999999997,"5min":6.016,"15min":6.016},"max":{"1min":49.905000000000001,"5min":203.38499999999999,"15min":203.38499999999999},"last":9.4860000000000007}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.252000000000001,"5min":15.499000000000001,"15min":16.039999999999999},"min":{"1min":6.6790000000000003,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":20.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":9.7759999999999998},{"interface":"front","average":{"1min":11.571999999999999,"5min":14.185,"15min":14.621},"min":{"1min":8.8179999999999996,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":16.605,"5min":37.390999999999998,"15min":37.390999999999998},"last":12.156000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.577999999999999,"5min":17.576000000000001,"15min":18.742000000000001},"min":{"1min":6.9020000000000001,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":14.788,"5min":183.38900000000001,"15min":183.38900000000001},"last":10.526},{"interface":"front","average":{"1min":14.112,"5min":21.276,"15min":22.469999999999999},"min":{"1min":8.2420000000000009,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.007000000000001,"5min":189.374,"15min":189.374},"last":17.986999999999998}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.077999999999999,"5min":20.582000000000001,"15min":21.5},"min":{"1min":11.079000000000001,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":27.879999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":11.278},{"interface":"front","average":{"1min":12.926,"5min":18.677,"15min":19.635000000000002},"min":{"1min":7.3929999999999998,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":26.975999999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":7.1760000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.343,"5min":25.315999999999999,"15min":26.812000000000001},"min":{"1min":10.385,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":21.745000000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":13.323},{"interface":"front","average":{"1min":15.525,"5min":24.969000000000001,"15min":26.542999999999999},"min":{"1min":10.387,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":19.696999999999999,"5min":187.863,"15min":187.863},"last":15.271000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.364000000000001,"5min":24.882000000000001,"15min":26.468},"min":{"1min":11.069000000000001,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":20.021999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":12.733000000000001},{"interface":"front","average":{"1min":18.702000000000002,"5min":28.018999999999998,"15min":29.571999999999999},"min":{"1min":11.772,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":25.587,"5min":275.92200000000003,"15min":275.92200000000003},"last":17.222999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":17.663,"5min":25.314,"15min":26.588999999999999},"min":{"1min":13.518000000000001,"5min":13.518000000000001,"15min":13.518000000000001},"max":{"1min":24.042999999999999,"5min":170.43600000000001,"15min":170.43600000000001},"last":16.341999999999999},{"interface":"front","average":{"1min":17.009,"5min":22.231999999999999,"15min":23.102},"min":{"1min":10.459,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":24.635000000000002,"5min":157.00200000000001,"15min":157.00200000000001},"last":14.609}]}]},{"osd":0,"up_from":16,"seq":68719476766,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.038,"5min":10.962999999999999,"15min":10.784000000000001},"min":{"1min":4.5149999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":39.700000000000003,"5min":39.700000000000003,"15min":39.700000000000003},"last":9.0980000000000008},{"interface":"front","average":{"1min":11.220000000000001,"5min":11.77,"15min":11.862},"min":{"1min":5.2050000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":28.358000000000001,"5min":30.788,"15min":30.788},"last":13.555}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.443,"5min":18.129999999999999,"15min":19.411000000000001},"min":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":20.760999999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":14.287000000000001},{"interface":"front","average":{"1min":12.442,"5min":14.385999999999999,"15min":14.710000000000001},"min":{"1min":6.8070000000000004,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":23.695,"5min":35.834000000000003,"15min":35.834000000000003},"last":11.036}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.311,"5min":20.981999999999999,"15min":22.094000000000001},"min":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":34.953000000000003,"5min":99.790999999999997,"15min":99.790999999999997},"last":8.1319999999999997},{"interface":"front","average":{"1min":14.433,"5min":16.257999999999999,"15min":16.562000000000001},"min":{"1min":9.5980000000000008,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":33.680999999999997,"5min":33.680999999999997,"15min":33.680999999999997},"last":11.731999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.423999999999999,"5min":15.603999999999999,"15min":16.300999999999998},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":19.856999999999999,"5min":41.134999999999998,"15min":41.134999999999998},"last":7.4550000000000001},{"interface":"front","average":{"1min":15.153,"5min":23.329000000000001,"15min":24.692},"min":{"1min":8.5579999999999998,"5min":8.5579999999999998,"15min":8.5579999999999998},"max":{"1min":23.863,"5min":103.705,"15min":103.705},"last":16.459}]},{"osd":5,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.125,"5min":18.056000000000001,"15min":18.710999999999999},"min":{"1min":8.4629999999999992,"5min":8.4629999999999992,"15min":8.4629999999999992},"max":{"1min":26.533999999999999,"5min":43.847000000000001,"15min":43.847000000000001},"last":10.356999999999999},{"interface":"front","average":{"1min":16.975999999999999,"5min":26.073,"15min":27.588999999999999},"min":{"1min":9.6270000000000007,"5min":9.6270000000000007,"15min":9.6270000000000007},"max":{"1min":28.102,"5min":98.707999999999998,"15min":98.707999999999998},"last":8.8000000000000007}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":18.401,"5min":20.082999999999998,"15min":20.363},"min":{"1min":8.7550000000000008,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":55.631,"5min":55.631,"15min":55.631},"last":12.339},{"interface":"front","average":{"1min":17.962,"5min":21.433,"15min":22.010999999999999},"min":{"1min":10.673999999999999,"5min":10.673999999999999,"15min":10.673999999999999},"max":{"1min":54.656999999999996,"5min":54.656999999999996,"15min":54.656999999999996},"last":16.77}]},{"osd":7,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":17.059000000000001,"5min":21.111000000000001,"15min":21.786000000000001},"min":{"1min":12.289,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":23.265000000000001,"5min":45.927999999999997,"15min":45.927999999999997},"last":12.911},{"interface":"front","average":{"1min":17.498999999999999,"5min":22.126000000000001,"15min":22.896999999999998},"min":{"1min":11.305999999999999,"5min":11.305999999999999,"15min":11.305999999999999},"max":{"1min":28.651,"5min":45.130000000000003,"15min":45.130000000000003},"last":17.859999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":9.0389999999999997,"5min":14.882999999999999,"15min":15.856999999999999},"min":{"1min":4.431,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":17.584,"5min":131.477,"15min":131.477},"last":4.5300000000000002},{"interface":"front","average":{"1min":9.7940000000000005,"5min":15.146000000000001,"15min":16.038},"min":{"1min":6.0839999999999996,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":20.175999999999998,"5min":129.386,"15min":129.386},"last":7.641}]},{"osd":1,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":10.300000000000001,"5min":10.855,"15min":10.946999999999999},"min":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"max":{"1min":14.603,"5min":25.774000000000001,"15min":25.774000000000001},"last":9.2940000000000005},{"interface":"front","average":{"1min":16.196000000000002,"5min":18.829999999999998,"15min":19.268999999999998},"min":{"1min":11.680999999999999,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":27.390999999999998,"5min":91.295000000000002,"15min":91.295000000000002},"last":16.581}]},{"osd":2,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.880000000000001,"5min":17.151,"15min":17.863},"min":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":46.811,"5min":134.21199999999999,"15min":134.21199999999999},"last":12.209},{"interface":"front","average":{"1min":14.634,"5min":14.634,"15min":14.634},"min":{"1min":5.992,"5min":5.992,"15min":5.992},"max":{"1min":50.450000000000003,"5min":50.450000000000003,"15min":50.450000000000003},"last":8.6980000000000004}]},{"osd":3,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.888,"5min":19.111999999999998,"15min":20.149999999999999},"min":{"1min":6.758,"5min":6.758,"15min":6.758},"max":{"1min":22.356999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":13.763999999999999},{"interface":"front","average":{"1min":15.807,"5min":20.111000000000001,"15min":20.829000000000001},"min":{"1min":9.1150000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":25.867000000000001,"5min":124.684,"15min":124.684},"last":15.597}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":12.601000000000001,"5min":13.630000000000001,"15min":13.802},"min":{"1min":8.5579999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":17.361999999999998,"5min":24.963000000000001,"15min":24.963000000000001},"last":9.5429999999999993},{"interface":"front","average":{"1min":16.216000000000001,"5min":15.724,"15min":15.641999999999999},"min":{"1min":11.066000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":26.544,"5min":26.544,"15min":26.544},"last":11.599}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.038,"5min":19.187000000000001,"15min":20.212},"min":{"1min":7.016,"5min":7.016,"15min":7.016},"max":{"1min":18.068999999999999,"5min":125.792,"15min":125.792},"last":10.782},{"interface":"front","average":{"1min":16.5,"5min":16.905000000000001,"15min":16.972999999999999},"min":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"max":{"1min":24.768000000000001,"5min":28.591999999999999,"15min":28.591999999999999},"last":14.877000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":16.189,"5min":28.972000000000001,"15min":31.102},"min":{"1min":7.6020000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":24.219000000000001,"5min":327.29599999999999,"15min":327.29599999999999},"last":17.373000000000001},{"interface":"front","average":{"1min":14.366,"5min":30.007999999999999,"15min":32.615000000000002},"min":{"1min":7.5499999999999998,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":18.904,"5min":323.82400000000001,"15min":323.82400000000001},"last":13.177}]}]},{"osd":5,"up_from":15,"seq":64424509471,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":590385,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.606999999999999,"5min":14.191000000000001,"15min":14.788},"min":{"1min":5.5609999999999999,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":28.821999999999999,"5min":68.513999999999996,"15min":68.513999999999996},"last":6.9749999999999996},{"interface":"front","average":{"1min":10.786,"5min":15.789999999999999,"15min":16.623999999999999},"min":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":27.027999999999999,"5min":88.816000000000003,"15min":88.816000000000003},"last":7.657}]},{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.936999999999999,"5min":16.495000000000001,"15min":17.254999999999999},"min":{"1min":7.3710000000000004,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":19.015999999999998,"5min":87.515000000000001,"15min":87.515000000000001},"last":37.406999999999996},{"interface":"front","average":{"1min":12.638999999999999,"5min":16.937999999999999,"15min":17.654},"min":{"1min":6.5549999999999997,"5min":6.21,"15min":6.21},"max":{"1min":26.303000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":38.204000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15.009,"5min":17.692,"15min":18.138999999999999},"min":{"1min":10.311,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":23.599,"5min":113.943,"15min":113.943},"last":9.1310000000000002},{"interface":"front","average":{"1min":11.092000000000001,"5min":14.593999999999999,"15min":15.178000000000001},"min":{"1min":5.7949999999999999,"5min":5.7949999999999999,"15min":5.7949999999999999},"max":{"1min":22.634,"5min":80.004000000000005,"15min":80.004000000000005},"last":12.994999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":16.504999999999999,"5min":20.773,"15min":21.484999999999999},"min":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":87.866,"5min":87.866,"15min":87.866},"last":9.6940000000000008},{"interface":"front","average":{"1min":19.504999999999999,"5min":23.062000000000001,"15min":23.655000000000001},"min":{"1min":11.359999999999999,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":89.334000000000003,"5min":91.183000000000007,"15min":91.183000000000007},"last":10.417}]},{"osd":4,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":12.736000000000001,"5min":12.737,"15min":12.737},"min":{"1min":7.9050000000000002,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":20.556999999999999,"5min":25.195,"15min":25.195},"last":13.592000000000001},{"interface":"front","average":{"1min":14.071,"5min":16.562999999999999,"15min":16.978000000000002},"min":{"1min":7.2279999999999998,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":19.228000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":14.31}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.146000000000001,"5min":15.726000000000001,"15min":15.99},"min":{"1min":7.6980000000000004,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":46.561999999999998,"5min":66.765000000000001,"15min":66.765000000000001},"last":40.939},{"interface":"front","average":{"1min":17.895,"5min":20.585000000000001,"15min":21.033000000000001},"min":{"1min":11.125999999999999,"5min":11.125999999999999,"15min":11.125999999999999},"max":{"1min":39.637999999999998,"5min":71.771000000000001,"15min":71.771000000000001},"last":39.588999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":17.015000000000001,"5min":18.721,"15min":19.004999999999999},"min":{"1min":10.923,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":25.045999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":8.3759999999999994},{"interface":"front","average":{"1min":15.677,"5min":17.84,"15min":18.199999999999999},"min":{"1min":8.1170000000000009,"5min":8.1170000000000009,"15min":8.1170000000000009},"max":{"1min":21.984000000000002,"5min":42.531999999999996,"15min":42.531999999999996},"last":14.968}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":192545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":8.6489999999999991,"5min":9.1769999999999996,"15min":9.2650000000000006},"min":{"1min":4.6660000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":26.984999999999999,"5min":26.984999999999999,"15min":26.984999999999999},"last":7.7960000000000003},{"interface":"front","average":{"1min":11.923999999999999,"5min":10.831,"15min":10.648999999999999},"min":{"1min":6.4500000000000002,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":25.934999999999999,"5min":25.934999999999999,"15min":25.934999999999999},"last":15.071}]},{"osd":1,"last update":"Mon Oct 30 06:26:20 2023","interfaces":[{"interface":"back","average":{"1min":12.436999999999999,"5min":11.664,"15min":11.535},"min":{"1min":6.9859999999999998,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":6.3929999999999998},{"interface":"front","average":{"1min":15.724,"5min":14.467000000000001,"15min":14.257},"min":{"1min":9.3049999999999997,"5min":6.609,"15min":6.609},"max":{"1min":30.751999999999999,"5min":30.751999999999999,"15min":30.751999999999999},"last":13.153}]},{"osd":2,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":14.199,"5min":12.414,"15min":12.116},"min":{"1min":7.4530000000000003,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":24.254999999999999,"5min":24.254999999999999,"15min":24.254999999999999},"last":10.564},{"interface":"front","average":{"1min":15.834,"5min":15.787000000000001,"15min":15.779},"min":{"1min":9.625,"5min":9.625,"15min":9.625},"max":{"1min":25.428000000000001,"5min":25.428000000000001,"15min":25.428000000000001},"last":14.311999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":10.042,"5min":11.02,"15min":11.183},"min":{"1min":6.415,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":32.927999999999997,"5min":32.927999999999997,"15min":32.927999999999997},"last":7.101},{"interface":"front","average":{"1min":14.454000000000001,"5min":15.743,"15min":15.958},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":19.853000000000002,"5min":30.285,"15min":30.285},"last":11.148999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.544,"5min":19.591000000000001,"15min":19.599},"min":{"1min":10.619999999999999,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":40.25,"5min":64.474999999999994,"15min":64.474999999999994},"last":15.811},{"interface":"front","average":{"1min":15.550000000000001,"5min":16.079000000000001,"15min":16.167000000000002},"min":{"1min":6.9809999999999999,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":37.826999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":9.2379999999999995}]},{"osd":6,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":20.481999999999999,"5min":19.119,"15min":18.891999999999999},"min":{"1min":14.273,"5min":14.273,"15min":14.273},"max":{"1min":35.241,"5min":35.241,"15min":35.241},"last":16.588999999999999},{"interface":"front","average":{"1min":14.154,"5min":14.051,"15min":14.034000000000001},"min":{"1min":8.8379999999999992,"5min":7.343,"15min":7.343},"max":{"1min":38.668999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":9.8219999999999992}]},{"osd":7,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":16.852,"5min":18.056000000000001,"15min":18.257000000000001},"min":{"1min":10.457000000000001,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":41.783999999999999,"5min":74.977999999999994,"15min":74.977999999999994},"last":12.558999999999999},{"interface":"front","average":{"1min":21.052,"5min":22.094000000000001,"15min":22.268000000000001},"min":{"1min":15.183999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":44.450000000000003,"5min":83.138000000000005,"15min":83.138000000000005},"last":17.457000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:26:30.614 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [1, 2] 2023-10-30T06:26:30.615 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '1', '2'] 2023-10-30T06:26:30.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 1 2 2023-10-30T06:26:30.651 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:30.652 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:30.650+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:30.739 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:30.739 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:30.741 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:30.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:30.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:30.842 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:30.841+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:30.877 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:30.877 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:30.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:30.945 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:30.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:31.014 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.014 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:31.044 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:31.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:31.144 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:31.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:31.198 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.198 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:31.250 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:31.247+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:31.360 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:31.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:31.374 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.375 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:31.447 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:31.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:31.548 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.548 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.549 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:31.547+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:31.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:31.649 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:31.648+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:31.725 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.726 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:31.777 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:31.775+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:31.851 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:31.850+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:31.905 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:31.906 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:31.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:31.952 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:31.951+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:31.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:26:32.053 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:32.052+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:32.088 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:32.088 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:32.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:32.154 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:32.153+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:32.255 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:32.254+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:32.275 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:32.275 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:32.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:32.356 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:32.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:32.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:32.414 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:32.414 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:32.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:32.456 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.5 pg_upmap mapping to [1,2] 2023-10-30T06:26:32.458 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:32.456+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:32.559 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:32.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:32.613 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:32.613 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:32.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:32.660 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:32.658+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:32.702 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:32.751 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:32.751 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:32.761 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:32.761 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:32.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:32.788 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:32.789 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:32.861 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:32.860+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:32.892 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:32.892 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:32.962 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:32.961+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:32.987 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:32.988 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:32.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:32.990 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:32.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:32.990 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:32.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.990 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:32.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:32.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 56.697892504999999, 2023-10-30T06:26:32.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:32.991 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:32.991 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:32.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:32.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:32.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:32.992 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:32.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:32.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:32.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:32.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:32.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:32.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:32.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:32.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:32.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:32.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:32.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:32.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:32.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:32.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:32.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:33.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:33.001 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:33.001 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:33.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:33.063 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:33.062+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:33.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:33.074 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:33.147 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:33.148 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:33.148 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:33.148 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:33.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:33.175 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:33.175 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:33.201 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:33.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:33.261 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:33.262 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:33.264 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:33.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:33.285 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:33.285 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:33.286 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:33.286 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:33.286 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:33.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:33.346 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:33.347 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:33.401 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:33.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:33.466 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:33.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:33.471 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:33.472 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:33.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:33.472 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:33.473 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:33.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:33.510 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:26:33.567 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:33.566+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:33.651 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:33.651 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:33.652 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:33.652 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:33.660 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:33.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:33.668 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:33.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:33.768 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:33.767+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:33.832 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:33.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:33.882 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:33.881+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:33.969 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:33.968+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:34.056 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:34.057 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:34.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:34.057 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:34.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.058 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:34.058 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:34.058 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 141.89448295099999, 2023-10-30T06:26:34.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:34.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:34.059 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:34.059 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:34.059 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:34.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:34.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.060 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:34.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:34.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:34.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:34.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:34.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:34.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:34.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:34.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:34.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:34.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:34.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:34.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:34.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:34.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:34.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:34.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.070 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.071 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.071 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:34.071 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:34.072 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 138.98130908799999, 2023-10-30T06:26:34.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:34.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.073 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:34.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:34.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:34.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:34.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:34.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:34.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:34.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:34.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:34.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:34.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:34.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:34.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:34.082 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.083 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.083 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:34.084 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:34.084 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 138.321131119, 2023-10-30T06:26:34.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:34.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.085 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:34.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:34.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:34.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:34.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:34.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:34.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:34.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:34.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:34.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:34.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:34.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:34.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:34.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:34.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:34.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:34.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.096 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:34.096 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:34.096 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 138.245707912, 2023-10-30T06:26:34.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:34.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.097 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:34.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:34.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:34.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:34.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:34.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:34.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:34.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:34.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:34.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:34.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:34.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:34.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:34.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:34.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:34.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:34.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.107 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.108 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:34.108 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:34.108 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.974584781, 2023-10-30T06:26:34.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:34.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.109 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:34.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:34.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:34.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:34.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:34.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:34.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:34.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:34.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:34.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:34.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:34.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:34.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:34.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.119 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.120 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:34.120 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:34.120 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.944305071, 2023-10-30T06:26:34.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:34.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.121 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:34.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:34.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:34.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:34.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:34.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:34.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:34.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:34.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:34.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.131 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:34.132 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:34.132 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.57039398699999, 2023-10-30T06:26:34.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:34.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.132 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.133 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:34.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:34.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:34.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:34.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:34.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:34.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:34.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:34.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:34.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:34.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:34.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:34.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:34.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:34.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.143 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.143 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.143 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:34.144 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:34.144 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.565895286, 2023-10-30T06:26:34.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:34.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:34.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:34.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:34.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:34.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:34.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:34.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:34.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.155 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:34.155 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:34.156 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.274291347, 2023-10-30T06:26:34.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:34.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.156 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:34.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:34.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:34.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:34.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:34.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:34.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:34.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:34.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:34.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:34.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:34.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:34.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:34.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.166 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.167 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:34.167 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:34.167 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.26835403000001, 2023-10-30T06:26:34.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:34.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.168 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:34.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:34.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:34.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:34.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:34.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:34.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:34.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:34.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:34.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:34.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:34.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:34.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.178 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:34.179 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:34.179 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.26705501500001, 2023-10-30T06:26:34.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:34.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.180 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:34.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:34.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:34.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:34.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:34.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:34.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:34.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:34.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:34.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:34.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:34.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:34.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:34.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:34.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:34.189 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.190 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.190 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.190 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:34.191 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:34.191 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.16406620699999, 2023-10-30T06:26:34.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:34.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:34.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:34.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:34.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:34.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:34.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:34.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:34.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:34.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:34.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:34.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:34.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:34.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:34.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:34.201 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.201 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.202 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:34.202 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:34.203 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.89216293999999, 2023-10-30T06:26:34.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:34.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.203 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:34.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:34.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:34.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:34.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:34.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:34.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:34.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:34.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:34.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:34.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:34.213 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.213 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.213 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.214 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:34.214 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:34.214 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.88466474800001, 2023-10-30T06:26:34.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:34.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.215 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:34.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:34.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:34.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:34.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:34.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:34.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:34.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:34.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:34.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:34.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:34.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:34.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:34.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:34.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.227 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:34.227 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:34.228 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.88400354300001, 2023-10-30T06:26:34.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:34.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.228 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:34.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:34.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:34.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:34.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:34.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:34.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:34.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:34.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:34.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:34.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:34.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:34.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:34.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:34.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.239 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:34.239 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:34.239 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.871417705, 2023-10-30T06:26:34.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:34.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.240 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:34.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:34.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:34.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:34.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:34.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:34.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:34.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:34.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:34.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:34.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:34.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:34.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:34.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.250 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.86532549200001, 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:34.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.252 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:34.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:34.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:34.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:34.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:34.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:34.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:34.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:34.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:34.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:34.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:34.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:34.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:34.261 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.262 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.262 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.262 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:34.263 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:34.263 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.86433015099999, 2023-10-30T06:26:34.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:34.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.264 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:34.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:34.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:34.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:34.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:34.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:34.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:34.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:34.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:34.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:34.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:34.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:34.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:34.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:34.273 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.274 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.274 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:34.275 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:34.275 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 129.54165757999999, 2023-10-30T06:26:34.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:34.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:34.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:34.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:34.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:34.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:34.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:34.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:34.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:34.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:34.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:34.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.286 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:34.286 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:34.287 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 129.50821609499999, 2023-10-30T06:26:34.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:34.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:34.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:34.287 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:34.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:34.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:34.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:34.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:34.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:34.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:34.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:34.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:34.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:34.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:34.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:34.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:34.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:34.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:34.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:34.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:34.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:34.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:34.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:34.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:34.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:34.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:34.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:34.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:34.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.297 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.297 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:34.298 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:34.298 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:34.299 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:34.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:34.301 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:34.298+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:34.401 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:34.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:34.462 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:36.062 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.472 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:34.473 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:34.473 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:34.473 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:34.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:34.502 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:34.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:34.507 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:36.107 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.538 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:36.138 94385== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.562 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:52.387 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.565 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:52.389 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.587 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:52.411 98448== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:34.603 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:34.601+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:34.650 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:34.650 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:34.650 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:34.651 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:34.651 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:34.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:34.703 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:34.702+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:34.804 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:34.803+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:34.829 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:34.830 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:34.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:34.830 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:34.830 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:34.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:34.921 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:34.920+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:35.006 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:35.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:35.008 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:35.008 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:35.008 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:35.009 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:35.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:35.108 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:35.106+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:35.206 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:35.207 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:35.207 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:35.207 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:35.208 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:35.210 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:35.208+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:35.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:35.310 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:35.309+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:35.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:35.401 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:35.402 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:35.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:35.402 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:35.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.403 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:35.403 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:35.403 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 58.429257618000001, 2023-10-30T06:26:35.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:35.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:35.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:35.404 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:35.404 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:35.404 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:35.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:35.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:35.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:35.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:35.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:35.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:35.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:35.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:35.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:35.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:35.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:35.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:35.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:35.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:35.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:35.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:35.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:35.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:35.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:35.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:35.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:35.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:35.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:35.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:35.413 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:35.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:35.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:35.413 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:35.414 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:35.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:35.416 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:35.414+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:35.516 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:35.515+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:35.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:26:35.569 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:35.570 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:35.570 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:35.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:35.617 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:35.616+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:35.630 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:35.631 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:35.719 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:35.717+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:35.740 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:35.741 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:35.742 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:35.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:35.820 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:35.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:35.884 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:35.884 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:35.893 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:35.894 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:35.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:35.921 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:35.919+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:35.972 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:35.972 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:36.032 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.032 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:36.058 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:36.059 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:26:36.059 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:36.057+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:36.123 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:36.122+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:36.149 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:36.150 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:26:36.173 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.173 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:36.224 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:36.223+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:36.235 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:36.235 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:26:36.320 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:36.320 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:26:36.324 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:36.322+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:36.356 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.356 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:36.425 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:36.423+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:36.526 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:36.524+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:36.533 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.533 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:36.627 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:36.625+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:36.708 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.709 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:36.728 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:36.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:36.829 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:36.828+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:36.894 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:36.894 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:36.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:36.931 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:36.929+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:37.032 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:37.030+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:37.033 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.033 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:37.133 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:37.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:37.182 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:26:37.219 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.220 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.229 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:37.271 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:37.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:37.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:37.357 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:37.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.399 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:37.454 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:37.452+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:37.475 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:26:37.476 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:26:37.537 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:37.536+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:37.556 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.556 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:37.638 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:37.636+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:37.739 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:37.738+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:37.751 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.751 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:37.840 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:37.838+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:37.941 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:37.940+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:37.945 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:37.945 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:37.949 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:37.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:38.042 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:38.041+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:38.143 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:38.142+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:38.145 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:38.145 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:38.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:38.160 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.164 96291== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:38.245 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:38.243+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:38.299 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:38.300 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:38.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:38.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:38.352 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:38.350+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:38.413 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.417 96291== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:38.445 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:38.444+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:38.493 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:38.493 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:38.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:38.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:38.567 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:38.586 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:38.585+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:38.647 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:38.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:38.649 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:38.649 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:38.671 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:38.671 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:38.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:38.732 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:38.733 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:38.748 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:38.747+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:38.818 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:38.819 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:38.850 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:38.849+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:38.859 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:38.860 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:38.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:38.914 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:38.914 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:38.951 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:38.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:38.999 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:38.999 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:39.000 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:39.000 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:39.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:39.052 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:39.050+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:39.090 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:39.090 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:39.153 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:39.152+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:39.189 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:39.190 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:39.193 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:39.193 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:39.194 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:39.194 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:39.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:26:39.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:39.286 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:39.285+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:39.354 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:39.353+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:39.373 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:39.373 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:39.373 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:39.373 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:39.374 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:39.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:39.467 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:39.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:39.553 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:39.553 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:39.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:39.554 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:39.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.554 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:39.554 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 62.580625468999997, 2023-10-30T06:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:39.555 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:39.556 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:39.556 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:39.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:39.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.556 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:39.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:39.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:39.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:39.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:39.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:39.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:39.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:39.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:39.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:39.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:39.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:39.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:39.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:39.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:39.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:39.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:39.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:39.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:39.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:39.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:39.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:39.565 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:39.565 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:39.565 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:39.567 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:39.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:39.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:39.668 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:39.667+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:39.700 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:39.700 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:39.700 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:39.700 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:39.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:39.769 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:39.768+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:39.839 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:39.840 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:39.840 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:39.840 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:39.840 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:39.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:39.870 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:39.870+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:39.972 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:39.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:39.975 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:39.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:39.976 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:39.976 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:39.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:40.073 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:40.072+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:40.121 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:40.121 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:40.121 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:40.121 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:40.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:40.174 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:40.173+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:40.262 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:40.262 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:40.263 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:40.263 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:40.263 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:40.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:40.274 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:40.274+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:40.375 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:40.375+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:40.494 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:40.494 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:40.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:40.495 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:40.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:40.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:40.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 147.15448276800001, 2023-10-30T06:26:40.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:40.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:40.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:40.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:40.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:40.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:40.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:40.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:40.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:40.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:40.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:40.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:40.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:40.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:40.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:40.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:40.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:40.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:40.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:40.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:40.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:40.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:40.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:40.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:40.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:40.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.515 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.515 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:40.515 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.83332757700001, 2023-10-30T06:26:40.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:40.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.516 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.517 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:40.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.517 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:40.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:40.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:40.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:40.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:40.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:40.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:40.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:40.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:40.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:40.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:40.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:40.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:40.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:40.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:40.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:40.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:40.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:40.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:40.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:40.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.82908302000001, 2023-10-30T06:26:40.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:40.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:40.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:40.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:40.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:40.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:40.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:40.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:40.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:40.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:40.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:40.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:40.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:40.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:40.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:40.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:40.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:40.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:40.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:40.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:40.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:40.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:40.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:40.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:40.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:40.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.551 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.551 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:40.551 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.743624595, 2023-10-30T06:26:40.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:40.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.552 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:40.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:40.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:40.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:40.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:40.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:40.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:40.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:40.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:40.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:40.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:40.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:40.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:40.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:40.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:40.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:40.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:40.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:40.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:40.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:40.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:40.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.570 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.570 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.47438137899999, 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.573 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:40.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.573 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:40.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:40.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:40.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:40.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:40.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:40.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:40.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:40.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:40.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:40.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:40.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:40.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:40.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:40.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:40.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:40.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:40.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:40.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.47354996799999, 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.588 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.589 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.589 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:40.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.590 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:40.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:40.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:40.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:40.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:40.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:40.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:40.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:40.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:40.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:40.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:40.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:40.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:40.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:40.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:40.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.603 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.603 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.604 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:40.604 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.472882841, 2023-10-30T06:26:40.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:40.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.604 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.605 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:40.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.606 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:40.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:40.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:40.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:40.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:40.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:40.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:40.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:40.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:40.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:40.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:40.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:40.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:40.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:40.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:40.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:40.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:40.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:40.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:40.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:40.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.620 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.621 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:40.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.45060877, 2023-10-30T06:26:40.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:40.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:40.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.623 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:40.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:40.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:40.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:40.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:40.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:40.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:40.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:40.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:40.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:40.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:40.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:40.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:40.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.449960914, 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:40.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.637 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:40.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.638 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:40.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:40.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:40.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:40.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:40.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:40.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:40.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:40.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:40.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:40.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:40.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:40.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:40.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:40.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:40.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:40.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:40.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 144.449274841, 2023-10-30T06:26:40.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:40.652 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:40.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.653 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:40.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:40.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:40.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:40.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:40.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:40.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:40.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:40.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:40.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:40.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:40.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:40.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:40.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:40.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:40.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:40.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.666 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:40.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:40.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 143.378392873, 2023-10-30T06:26:40.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:40.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:40.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:40.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:40.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:40.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:40.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:40.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:40.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:40.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:40.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:40.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:40.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:40.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:40.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:40.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 143.37772890700001, 2023-10-30T06:26:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:40.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:40.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:40.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:40.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:40.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:40.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:40.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:40.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:40.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:40.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:40.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:40.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:40.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:40.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:40.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:40.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:40.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:40.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.699 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:40.699 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:40.699 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 143.34927509799999, 2023-10-30T06:26:40.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:40.700 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.700 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.701 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.702 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:40.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.702 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:40.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:40.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:40.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:40.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:40.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:40.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:40.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:40.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:40.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:40.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:40.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:40.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:40.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:40.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:40.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:40.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:40.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:40.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:40.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:40.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:40.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:40.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:40.718 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.719 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:40.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 136.007508605, 2023-10-30T06:26:40.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:40.720 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.720 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.720 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.721 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:40.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.721 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:40.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:40.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:40.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:40.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:40.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:40.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:40.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:40.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:40.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:40.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:40.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:40.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:40.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:40.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:40.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:40.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:40.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:40.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:40.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:40.734 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.735 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.736 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:40.736 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 136.00675993499999, 2023-10-30T06:26:40.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:40.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:40.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:40.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:40.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:40.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:40.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:40.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:40.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:40.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:40.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:40.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:40.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:40.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:40.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:40.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:40.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:40.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:40.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.751 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 136.00609155199999, 2023-10-30T06:26:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:40.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.753 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:40.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.754 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:40.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:40.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:40.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:40.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:40.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:40.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:40.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:40.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:40.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:40.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:40.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:40.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:40.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:40.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:40.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:40.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:40.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:40.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:40.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:40.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.771 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.771 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.99624087800001, 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:40.772 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.773 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.773 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.773 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:40.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.774 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:40.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:40.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:40.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:40.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:40.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:40.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:40.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:40.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:40.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:40.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:40.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:40.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:40.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:40.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.786 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.787 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.787 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:40.787 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.99556706300001, 2023-10-30T06:26:40.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:40.788 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.788 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.788 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:40.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:40.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:40.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:40.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:40.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:40.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:40.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:40.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:40.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:40.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:40.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:40.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:40.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:40.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:40.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:40.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:40.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:40.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.803 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.99491868000001, 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:40.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:40.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:40.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:40.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:40.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:40.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:40.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:40.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:40.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:40.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:40.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:40.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:40.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:40.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:40.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:40.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:40.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 135.99080221400001, 2023-10-30T06:26:40.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:40.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:40.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:40.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:40.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:40.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:40.822 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:40.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.822 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:40.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:40.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:40.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:40.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:40.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:40.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:40.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:40.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:40.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:40.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:40.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:40.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:40.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:40.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:40.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:40.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:40.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:40.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:40.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:40.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:40.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:40.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:40.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:40.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:40.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:40.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:40.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:40.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:40.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:40.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:40.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:40.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:40.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:40.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:40.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.840 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:40.841 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:40.842 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:40.843 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:40.494+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:40.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:40.845 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:26:40.845 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:40.842+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:40.945 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:40.943+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:40.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:41.013 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:41.014 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:41.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:41.068 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:41.066+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:41.148 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:41.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:41.176 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:41.176 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:41.176 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:41.177 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:41.177 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:41.185 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:41.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:41.269 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:41.268+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:41.347 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:41.345+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:41.382 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:41.383 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:41.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:41.383 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:41.383 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:41.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:41.448 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:41.448+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:41.480 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:41.480 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.549 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:41.549+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:41.589 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:41.590 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.594 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:41.594 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:41.595 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:41.595 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:41.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:41.650 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:41.649+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:41.663 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:41.681 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:41.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.751 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:41.750+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:41.781 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:41.781 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.782 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:41.782 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:41.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:41.852 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:41.852+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:41.864 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.954 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:41.952+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:41.963 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:41.963 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:26:41.970 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:41.971 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:41.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:42.048 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:42.048 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:26:42.054 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:42.054+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:42.109 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.110 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:42.133 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:42.133 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:26:42.161 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:42.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:42.271 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:42.270+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:42.284 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.284 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:42.357 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:42.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:42.458 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:42.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:42.459 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.459 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:42.478 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:26:42.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-10-30T06:26:42.559 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:42.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:42.596 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.596 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:42.660 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:42.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.733 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:42.761 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:42.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:42.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:26:42.891 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:42.889+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:42.908 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:42.908 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:42.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:42.962 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:42.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:43.044 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.044 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:43.063 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:43.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:43.164 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:43.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:43.219 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.219 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:43.264 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:43.262+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:43.365 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:43.363+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:43.395 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.395 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:43.466 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:43.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:43.528 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.529 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:43.581 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:43.580+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:43.666 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.667 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.669 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:43.667+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:43.676 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:43.770 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:43.768+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:43.841 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:43.842 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:43.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:43.870 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:43.869+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:43.971 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:43.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2023-10-30T06:26:43.978 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:26:43.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-10-30T06:26:44.002 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:44.003 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:44.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:44.072 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:44.071+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:44.146 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:44.173 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:44.172+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:44.189 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:44.190 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:44.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:44.288 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:44.286+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:44.375 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:44.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:44.414 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:44.414 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.416 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:44.416 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:44.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:44.416 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:44.417 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:44.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:44.476 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:44.474+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:44.505 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:44.505 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.576 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:44.575+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:44.617 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:44.617 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:44.617 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:44.618 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:44.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:44.635 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:44.635 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:44.680 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:44.677+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:44.734 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:44.735 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.776 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:44.776 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:44.777 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:44.777 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:44.777 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:44.777 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:44.775+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:44.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:44.818 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.878 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:44.876+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:44.901 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:44.901 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:44.979 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:44.977+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:44.984 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:44.985 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:45.053 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:45.054 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:45.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:45.054 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:45.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 151.68145376000001, 2023-10-30T06:26:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:45.055 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:45.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:45.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:45.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:45.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:45.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:45.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:45.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:45.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:45.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:45.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:45.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:45.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:45.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:45.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:45.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:45.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:45.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:45.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:45.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:45.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:45.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:45.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:45.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:45.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:45.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:45.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.074 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.074 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.075 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:45.075 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.36029856900001, 2023-10-30T06:26:45.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:45.075 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.075 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:45.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:45.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:45.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:45.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:45.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:45.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:45.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:45.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:45.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:45.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:45.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:45.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:45.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:45.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:45.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:45.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:45.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:45.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:45.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.089 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.35605401199999, 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.092 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:45.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:45.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:45.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:45.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:45.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:45.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:45.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:45.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:45.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:45.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:45.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:45.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:45.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:45.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:45.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:45.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:45.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:45.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:45.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.106 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.106 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:45.106 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.270595587, 2023-10-30T06:26:45.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:45.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.107 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.107 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.107 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:45.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:45.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:45.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:45.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:45.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:45.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:45.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:45.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:45.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:45.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:45.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:45.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:45.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.121 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:45.122 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.001352371, 2023-10-30T06:26:45.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:45.122 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.122 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.122 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.123 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.123 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:45.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.123 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:45.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:45.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:45.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:45.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:45.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:45.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:45.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:45.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:45.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:45.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:45.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:45.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:45.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:45.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:45.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:45.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:45.136 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.136 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.137 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.137 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.138 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:45.138 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.00052095999999, 2023-10-30T06:26:45.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:45.138 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.138 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.139 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.139 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:45.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.140 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:45.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:45.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:45.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:45.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:45.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:45.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:45.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:45.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:45.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:45.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:45.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:45.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:45.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:45.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:45.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:45.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:45.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.153 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.154 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.155 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:45.155 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.999853833, 2023-10-30T06:26:45.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:45.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.155 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:45.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.157 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:45.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:45.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:45.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:45.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:45.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:45.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:45.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:45.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:45.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:45.169 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.170 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.977579762, 2023-10-30T06:26:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:45.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:45.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:45.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:45.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:45.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:45.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:45.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:45.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:45.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:45.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:45.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:45.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.976931906, 2023-10-30T06:26:45.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:45.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.97624583300001, 2023-10-30T06:26:45.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.206 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:45.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:45.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:45.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:45.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:45.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:45.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:45.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:45.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 147.905363865, 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:45.221 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.222 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:45.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:45.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:45.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:45.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:45.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:45.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:45.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:45.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:45.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 147.90469989900001, 2023-10-30T06:26:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:45.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.239 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:45.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:45.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:45.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:45.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 147.87624609, 2023-10-30T06:26:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:45.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:45.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:45.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:45.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:45.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:45.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:45.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:45.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:45.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.271 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.534479597, 2023-10-30T06:26:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.272 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.273 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.273 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:45.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.274 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:45.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:45.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:45.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:45.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:45.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:45.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:45.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:45.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:45.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:45.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:45.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:45.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:45.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:45.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:45.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:45.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.288 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:45.289 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.53373092699999, 2023-10-30T06:26:45.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:45.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.290 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:45.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.291 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:45.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:45.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:45.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:45.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:45.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:45.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:45.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:45.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:45.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:45.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:45.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:45.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:45.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:45.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:45.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:45.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:45.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:45.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:45.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:45.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:45.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.53306254399999, 2023-10-30T06:26:45.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:45.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:45.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:45.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:45.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:45.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:45.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:45.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:45.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:45.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:45.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:45.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:45.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:45.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:45.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:45.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:45.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:45.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:45.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:45.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:45.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:45.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:45.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:45.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:45.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:45.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:45.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.52321187000001, 2023-10-30T06:26:45.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.52253805500001, 2023-10-30T06:26:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.52188967199999, 2023-10-30T06:26:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.51777320599999, 2023-10-30T06:26:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:45.391 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:45.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:45.394 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:45.395 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:45.396 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:45.396 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:45.392+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:45.494 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:45.493+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:45.574 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:45.575 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:45.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:45.595 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:45.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:45.601 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2023-10-30T06:26:45.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-10-30T06:26:45.696 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:45.694+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:45.730 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:45.731 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:45.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:45.796 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:45.795+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:45.890 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:45.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:45.891 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:45.897 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:45.895+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:45.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:45.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:26:45.998 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:45.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:46.072 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:46.072 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:46.073 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:46.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:46.098 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:46.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:46.200 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:46.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:46.216 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:46.216 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:46.217 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:46.217 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:46.217 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:46.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:46.312 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:46.311+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:46.401 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:46.402 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 70.109304354000002, 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:46.403 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:46.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:46.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:46.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:46.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:46.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:46.413 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:46.413 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:46.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:46.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:46.503 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:46.501+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:46.541 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:46.541 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:46.541 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:46.541 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:46.549 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:46.603 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:46.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:46.667 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2023-10-30T06:26:46.673 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:46.674 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:46.674 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:46.674 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:46.674 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:46.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-10-30T06:26:46.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:46.723 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:46.720+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:46.805 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:46.803+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:46.807 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:46.807 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:46.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:46.808 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:46.808 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:46.816 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:46.905 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:46.904+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:46.983 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:46.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:46.984 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:46.984 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:46.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:47.034 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:47.033+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:47.107 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:47.105+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:47.159 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:47.159 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:47.159 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:47.160 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:47.160 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:47.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:47.212 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:47.211+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:47.308 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:47.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:47.334 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:47.334 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:47.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:47.335 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.17865541099999, 2023-10-30T06:26:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:47.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:47.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:47.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:47.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:47.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:47.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:47.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:47.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:47.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:47.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:47.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:47.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.348 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:47.348 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:47.348 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 152.265481548, 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:47.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:47.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:47.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:47.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:47.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:47.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:47.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:47.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:47.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:47.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:47.359 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.359 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.359 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 151.60530357900001, 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:47.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:47.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:47.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:47.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:47.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:47.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:47.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:47.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:47.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:47.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:47.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:47.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.371 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 151.52988037200001, 2023-10-30T06:26:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.373 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:47.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:47.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:47.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:47.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:47.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:47.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:47.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:47.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:47.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:47.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:47.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.383 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 151.25875724100001, 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:47.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:47.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:47.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.394 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:47.395 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 151.22847753100001, 2023-10-30T06:26:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.396 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:47.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:47.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:47.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:47.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:47.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:47.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:47.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:47.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:47.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:47.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:47.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:47.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.406 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:47.407 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:47.408 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.854566447, 2023-10-30T06:26:47.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:47.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.408 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:47.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:47.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:47.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:47.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:47.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:47.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:47.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:47.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:47.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:47.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.418 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.85006774600001, 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:47.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.420 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:47.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:47.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:47.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:47.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:47.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:47.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:47.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:47.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:47.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.430 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.55846380700001, 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:47.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.432 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:47.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:47.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:47.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:47.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:47.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:47.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:47.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:47.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:47.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:47.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:47.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:47.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:47.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.442 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.443 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:47.443 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:47.443 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.55252648999999, 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:47.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:47.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:47.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:47.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:47.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:47.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:47.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:47.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:47.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:47.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:47.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:47.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:47.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:47.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:47.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:47.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:47.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:47.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.455 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.456 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:47.456 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:47.456 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.55122747499999, 2023-10-30T06:26:47.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:47.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.457 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:47.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:47.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:47.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:47.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:47.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:47.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:47.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:47.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:47.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:47.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:47.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:47.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:47.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:47.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:47.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.467 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.468 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:47.468 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:47.468 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.448238667, 2023-10-30T06:26:47.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:47.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:47.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:47.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:47.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:47.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:47.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:47.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:47.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:47.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:47.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:47.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:47.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:47.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:47.479 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:47.480 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.1763354, 2023-10-30T06:26:47.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:47.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.480 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:47.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:47.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:47.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:47.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:47.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.490 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.490 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.16883720800001, 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.491 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:47.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:47.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:47.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:47.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:47.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:47.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:47.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:47.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:47.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.502 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.502 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.502 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:47.503 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:47.503 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.16817600300001, 2023-10-30T06:26:47.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:47.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:47.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:47.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:47.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:47.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:47.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:47.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:47.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:47.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:47.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:47.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:47.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.513 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:47.514 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.15559016500001, 2023-10-30T06:26:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:47.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:47.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:47.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:47.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:47.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:47.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:47.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:47.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:47.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:47.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:47.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:47.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.526 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:47.526 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:47.526 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.14949795199999, 2023-10-30T06:26:47.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:47.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.527 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:47.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:47.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:47.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:47.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:47.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:47.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:47.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:47.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:47.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:47.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:47.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:47.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:47.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:47.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:47.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:47.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.537 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.538 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:47.538 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:47.538 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 150.148502611, 2023-10-30T06:26:47.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:47.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:47.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 142.82583004, 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.551 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:47.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:47.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:47.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:47.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:47.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:47.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.560 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:47.561 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:47.561 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 142.792388555, 2023-10-30T06:26:47.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:47.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:47.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:47.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:47.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:47.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:47.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:47.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:47.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:47.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:47.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:47.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:47.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:47.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:47.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:47.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:47.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:47.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:47.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:47.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:47.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:47.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:47.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:47.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:47.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.571 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:47.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:47.572 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:47.573 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:47.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:47.575 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:47.573+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:47.675 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:47.674+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:47.742 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:47.743 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:47.746 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2023-10-30T06:26:47.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:47.752 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:26:47.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-10-30T06:26:47.792 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:47.791+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:47.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:47.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:47.876 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:47.875+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:47.918 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:47.919 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:47.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:47.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:47.933 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:47.977 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:47.975+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:48.056 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:48.056 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.079 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:48.078+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:48.094 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:48.095 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:48.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:48.143 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:48.144 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.179 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:48.179+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:48.229 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:48.230 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.272 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:48.272 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:48.272 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:48.272 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:48.280 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:48.279+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:48.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:48.333 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:48.333 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.381 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:48.380+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:48.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:48.408 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:48.408 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:48.408 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:48.408 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:48.408 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:48.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:48.437 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:48.438 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.481 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:48.480+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:48.525 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:48.525 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:26:48.561 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:48.561 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:48.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:48.621 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:48.620+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:48.682 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:48.682+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:48.705 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:48.705 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:48.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:48.799 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:48.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:48.884 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:48.882+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:48.924 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:48.924 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:48.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:48.941 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2023-10-30T06:26:48.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2023-10-30T06:26:49.000 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:48.998+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:49.071 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.071 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:49.086 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:49.086 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:49.086+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:49.199 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:49.198+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:49.213 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.213 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:49.308 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:49.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:49.391 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:49.389+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:49.403 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.403 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:49.493 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:49.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:49.594 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.595 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.595 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:49.592+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:49.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:49.691 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:49.690+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:49.748 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.748 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.758 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:49.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:26:49.791 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:49.790+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:49.892 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:49.891+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:49.938 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:49.938 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:49.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:49.994 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:49.992+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:50.079 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.080 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:50.094 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:50.093+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:50.196 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:50.195+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:50.217 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.218 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:50.300 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:50.298+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:50.398 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:50.397+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:50.404 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.404 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:50.500 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:50.498+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:50.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:26:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2023-10-30T06:26:50.601 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:50.600+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:50.601 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.601 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-10-30T06:26:50.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:50.703 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:50.703+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:50.744 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.744 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:50.789 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:50.790 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:50.802 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:50.801+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:50.877 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:50.878 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:50.882 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:50.883 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:50.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:50.917 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:50.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:50.970 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:50.971 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.004 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:51.004+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:51.060 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:51.061 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:51.063 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:51.063 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:51.104 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:51.104+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:51.150 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:51.150 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.205 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:51.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:51.239 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:51.239 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.240 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:51.240 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:51.240 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:51.241 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:51.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:51.324 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:51.324 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.333 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:51.331+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:51.407 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:51.407+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:51.416 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:51.416 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:51.433 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:51.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:51.433 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:51.434 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:51.434 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:51.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:51.527 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:51.525+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:51.610 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:51.608+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:51.647 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:51.647 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:51.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:51.648 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:51.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.648 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:26:51.648 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:51.649 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 159.49189091, 2023-10-30T06:26:51.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:26:51.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:51.650 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:51.650 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:26:51.650 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:51.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:26:51.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:26:51.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:26:51.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:26:51.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:26:51.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:26:51.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:26:51.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:26:51.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:26:51.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:26:51.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:26:51.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:26:51.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:26:51.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:26:51.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:26:51.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:26:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:26:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:51.661 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 156.578717047, 2023-10-30T06:26:51.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:26:51.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.662 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:51.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:26:51.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:26:51.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:26:51.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:26:51.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:26:51.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:26:51.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:26:51.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:26:51.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:26:51.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:26:51.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:26:51.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:26:51.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:26:51.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.673 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:26:51.673 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:51.673 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.91853907800001, 2023-10-30T06:26:51.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:26:51.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.674 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:51.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:26:51.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:26:51.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:26:51.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:26:51.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:26:51.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:26:51.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:26:51.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.84311587100001, 2023-10-30T06:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.57199274000001, 2023-10-30T06:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:26:51.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:26:51.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:26:51.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:26:51.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:26:51.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:26:51.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:26:51.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:26:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:26:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:26:51.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:26:51.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:26:51.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:26:51.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.707 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.708 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:26:51.708 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:51.708 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.54171303000001, 2023-10-30T06:26:51.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:26:51.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:51.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:26:51.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:26:51.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:26:51.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:26:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:26:51.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:26:51.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:26:51.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:26:51.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:26:51.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:26:51.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:26:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:26:51.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.719 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.719 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:26:51.719 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.167801946, 2023-10-30T06:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:26:51.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:26:51.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:26:51.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:26:51.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:26:51.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:26:51.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:26:51.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:26:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:26:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:26:51.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:26:51.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:26:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:26:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.730 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.731 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:26:51.731 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:51.731 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.16330324500001, 2023-10-30T06:26:51.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:26:51.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.732 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:26:51.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.87169930600001, 2023-10-30T06:26:51.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:26:51.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:26:51.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:26:51.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:26:51.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:26:51.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:26:51.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:26:51.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:26:51.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:26:51.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:26:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.755 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:26:51.755 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:51.755 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.86576198899999, 2023-10-30T06:26:51.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:26:51.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:26:51.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:26:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:26:51.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:26:51.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:26:51.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:26:51.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:26:51.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:26:51.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:26:51.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:26:51.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:26:51.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:26:51.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:26:51.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:26:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:26:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.766 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.766 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.766 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.86446297399999, 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.767 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:51.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:26:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:26:51.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:26:51.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:26:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:26:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:26:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:26:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:26:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:26:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:26:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:26:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:26:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:26:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:26:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:26:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.777 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:51.778 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.761474166, 2023-10-30T06:26:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:26:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:26:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:26:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:26:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:26:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:26:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:26:51.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:26:51.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:26:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:26:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:26:51.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:26:51.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.790 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:26:51.790 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:51.790 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.489570899, 2023-10-30T06:26:51.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:26:51.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.791 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:26:51.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:26:51.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:26:51.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:26:51.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:26:51.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:26:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:26:51.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:26:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:26:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:26:51.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:26:51.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:26:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:26:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.800 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.801 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.801 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:26:51.801 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:51.802 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.48207270699999, 2023-10-30T06:26:51.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:26:51.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.802 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:51.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:26:51.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:26:51.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:26:51.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:26:51.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:26:51.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:26:51.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:26:51.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:26:51.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:26:51.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:26:51.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:26:51.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:26:51.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.815 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.816 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:26:51.816 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:51.816 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.48141150199999, 2023-10-30T06:26:51.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:51.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:26:51.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:26:51.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:26:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:26:51.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:26:51.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:26:51.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:26:51.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:26:51.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:26:51.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:26:51.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:26:51.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:26:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:26:51.826 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:26:51.827 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:51.828 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.46882566400001, 2023-10-30T06:26:51.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:26:51.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.828 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:26:51.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:26:51.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:26:51.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:26:51.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:26:51.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:26:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:26:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:26:51.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:26:51.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:26:51.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:26:51.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:26:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:26:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.839 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:26:51.839 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:51.839 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.46273345099999, 2023-10-30T06:26:51.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:26:51.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.840 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:51.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:26:51.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:26:51.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:26:51.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:26:51.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:26:51.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:26:51.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:26:51.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:26:51.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:26:51.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:26:51.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.850 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.850 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.850 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:26:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.46173811, 2023-10-30T06:26:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:26:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:26:51.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:26:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:26:51.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:26:51.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:26:51.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:26:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:26:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:26:51.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:26:51.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:26:51.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:26:51.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.862 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:26:51.862 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:51.862 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 147.139065539, 2023-10-30T06:26:51.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:26:51.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.863 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:26:51.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:26:51.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:26:51.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:26:51.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:26:51.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:26:51.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:26:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:26:51.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:26:51.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:26:51.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:26:51.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:26:51.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:26:51.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:26:51.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.873 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:26:51.874 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:51.874 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 147.105624054, 2023-10-30T06:26:51.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:26:51.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:51.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:51.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:51.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:26:51.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:26:51.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:26:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:26:51.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:51.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:26:51.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:26:51.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:51.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:26:51.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:26:51.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:26:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:26:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:26:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:26:51.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:26:51.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:26:51.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:51.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:51.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:51.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:26:51.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:26:51.884 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.884 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:51.885 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:51.885 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:51.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:51.887 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:51.886+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:51.987 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:51.987+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:52.054 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.054 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:52.054 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:52.054 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:52.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:52.107 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:52.106+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:52.189 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:52.188+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:52.192 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2023-10-30T06:26:52.194 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.194 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:52.194 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:52.195 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:52.195 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:52.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-10-30T06:26:52.290 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:52.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:52.380 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:26:52.381 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:52.391 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:52.391+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:52.493 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:52.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:52.516 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:52.516 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:52.594 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:52.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:52.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:26:52.694 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:52.695 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.695 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:52.693+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:52.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:52.795 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:52.794+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:52.826 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:52.870 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:52.870 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:26:52.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 75.897828337999997, 2023-10-30T06:26:52.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:26:52.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:26:52.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:26:52.872 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:26:52.872 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:26:52.872 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:52.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:26:52.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.873 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:26:52.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:26:52.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:52.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:52.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:26:52.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:26:52.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:26:52.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:26:52.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:26:52.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:26:52.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:26:52.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:26:52.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:26:52.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:26:52.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:26:52.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:26:52.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:26:52.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:26:52.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:26:52.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:26:52.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:26:52.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:26:52.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:26:52.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:26:52.881 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:52.881 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:52.881 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:52.882 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:26:52.882 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:26:52.882 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:52.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:52.928 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:52.927+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:52.997 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:52.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:53.021 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:53.022 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:53.022 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:53.022 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:53.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:53.120 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:53.119+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:53.199 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:53.197+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:53.206 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:53.206 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:53.206 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:53.206 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:53.207 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:53.216 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:53.272 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2023-10-30T06:26:53.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-10-30T06:26:53.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:26:53.329 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:53.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:53.400 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:53.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:53.401 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:53.401 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:53.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:53.402 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:53.402 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:53.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:53.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:53.501 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:53.500+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:53.571 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:53.571 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:53.572 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:53.572 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:53.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:53.602 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:53.601+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:53.722 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:53.722 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:53.723 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:53.721+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:53.752 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:53.752 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:53.752 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:53.752 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:53.753 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:53.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:53.804 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:53.803+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:53.836 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:53.836 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:53.905 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:53.904+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:53.942 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:53.942 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:53.942 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:53.943 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:53.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:53.964 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:53.965 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:26:53.982 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2023-10-30T06:26:53.284816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221000,"kb_used_data":4828,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753720,"statfs":{"total":773094113280,"available":772867809280,"internally_reserved":0,"allocated":4943872,"data_stored":2773375,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.402200"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.295296+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:26:50.295296+0000","last_peered":"2023-10-30T06:26:50.295296+0000","last_clean":"2023-10-30T06:26:50.295296+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:26:50.295296+0000","last_undegraded":"2023-10-30T06:26:50.295296+0000","last_fullsized":"2023-10-30T06:26:50.295296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.980342+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:26:48.980342+0000","last_peered":"2023-10-30T06:26:48.980342+0000","last_clean":"2023-10-30T06:26:48.980342+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:26:48.980342+0000","last_undegraded":"2023-10-30T06:26:48.980342+0000","last_fullsized":"2023-10-30T06:26:48.980342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-30T06:26:44.564977+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:26:44.564977+0000","last_peered":"2023-10-30T06:26:44.564977+0000","last_clean":"2023-10-30T06:26:44.564977+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:26:44.564977+0000","last_undegraded":"2023-10-30T06:26:44.564977+0000","last_fullsized":"2023-10-30T06:26:44.564977+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.899988+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:26:48.899988+0000","last_peered":"2023-10-30T06:26:48.899988+0000","last_clean":"2023-10-30T06:26:48.899988+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:26:48.899988+0000","last_undegraded":"2023-10-30T06:26:48.899988+0000","last_fullsized":"2023-10-30T06:26:48.899988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.290952+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:26:50.290952+0000","last_peered":"2023-10-30T06:26:50.290952+0000","last_clean":"2023-10-30T06:26:50.290952+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:26:50.290952+0000","last_undegraded":"2023-10-30T06:26:50.290952+0000","last_fullsized":"2023-10-30T06:26:50.290952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.393598+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:26:49.393598+0000","last_peered":"2023-10-30T06:26:49.393598+0000","last_clean":"2023-10-30T06:26:49.393598+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:26:49.393598+0000","last_undegraded":"2023-10-30T06:26:49.393598+0000","last_fullsized":"2023-10-30T06:26:49.393598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.286753+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:26:50.286753+0000","last_peered":"2023-10-30T06:26:50.286753+0000","last_clean":"2023-10-30T06:26:50.286753+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:26:50.286753+0000","last_undegraded":"2023-10-30T06:26:50.286753+0000","last_fullsized":"2023-10-30T06:26:50.286753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.395336+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:26:49.395336+0000","last_peered":"2023-10-30T06:26:49.395336+0000","last_clean":"2023-10-30T06:26:49.395336+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:26:49.395336+0000","last_undegraded":"2023-10-30T06:26:49.395336+0000","last_fullsized":"2023-10-30T06:26:49.395336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":145,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-30T06:26:44.560201+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:26:44.560201+0000","last_peered":"2023-10-30T06:26:44.560201+0000","last_clean":"2023-10-30T06:26:44.560201+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:26:44.560201+0000","last_undegraded":"2023-10-30T06:26:44.560201+0000","last_fullsized":"2023-10-30T06:26:44.560201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411363,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":629133,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":11.029999999999999,"5min":10.494,"15min":10.404},"min":{"1min":5.2720000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"last":9.2889999999999997},{"interface":"front","average":{"1min":12.122999999999999,"5min":12.090999999999999,"15min":12.085000000000001},"min":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":35.398000000000003,"5min":46.798000000000002,"15min":46.798000000000002},"last":8.9209999999999994}]},{"osd":2,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":13.641,"15min":13.744999999999999},"min":{"1min":5.7350000000000003,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":44.537999999999997,"5min":52.552,"15min":52.552},"last":20.416},{"interface":"front","average":{"1min":17.143000000000001,"5min":15.545999999999999,"15min":15.279999999999999},"min":{"1min":7.391,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":55.585999999999999,"5min":55.585999999999999,"15min":55.585999999999999},"last":21.369}]},{"osd":3,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":15.367000000000001,"5min":21.626000000000001,"15min":22.669},"min":{"1min":8.5980000000000008,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":32.259999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":13.352},{"interface":"front","average":{"1min":14.832000000000001,"5min":21.024000000000001,"15min":22.056000000000001},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":38.987000000000002,"5min":124.075,"15min":124.075},"last":12.323}]},{"osd":4,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":13.747,"15min":13.699},"min":{"1min":7.6470000000000002,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":25.023,"5min":51.649000000000001,"15min":51.649000000000001},"last":9.7859999999999996},{"interface":"front","average":{"1min":18.245000000000001,"5min":18.047999999999998,"15min":18.015000000000001},"min":{"1min":10.759,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":44.973999999999997,"5min":44.973999999999997,"15min":44.973999999999997},"last":15.242000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":16.564,"5min":20.154,"15min":20.751999999999999},"min":{"1min":7.351,"5min":7.351,"15min":7.351},"max":{"1min":58.648000000000003,"5min":123.754,"15min":123.754},"last":10.574999999999999},{"interface":"front","average":{"1min":14.667999999999999,"5min":21.396999999999998,"15min":22.518999999999998},"min":{"1min":7.6619999999999999,"5min":7.6619999999999999,"15min":7.6619999999999999},"max":{"1min":41.558999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":16.882999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":17.972000000000001,"5min":20.760000000000002,"15min":21.224},"min":{"1min":9.359,"5min":9.359,"15min":9.359},"max":{"1min":39.860999999999997,"5min":66.400999999999996,"15min":66.400999999999996},"last":16.16},{"interface":"front","average":{"1min":16.838999999999999,"5min":19.254999999999999,"15min":19.657},"min":{"1min":9.6969999999999992,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":37.027999999999999,"5min":53.149000000000001,"15min":53.149000000000001},"last":13.975}]},{"osd":7,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":18.468,"5min":20.904,"15min":21.309999999999999},"min":{"1min":10.083,"5min":10.083,"15min":10.083},"max":{"1min":51.375,"5min":51.375,"15min":51.375},"last":17.675000000000001},{"interface":"front","average":{"1min":19.795999999999999,"5min":21.234999999999999,"15min":21.475000000000001},"min":{"1min":10.004,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":54.439,"5min":69.991,"15min":69.991},"last":19.640000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476773,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.561,"5min":13.877000000000001,"15min":14.429},"min":{"1min":4.9980000000000002,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":37.701999999999998,"5min":94.343000000000004,"15min":94.343000000000004},"last":14.741},{"interface":"front","average":{"1min":11.94,"5min":14.991,"15min":15.499000000000001},"min":{"1min":5.8700000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":49.045999999999999,"5min":122.054,"15min":122.054},"last":5.6390000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.071,"5min":10.667,"15min":10.766},"min":{"1min":5.8789999999999996,"5min":5.8789999999999996,"15min":5.8789999999999996},"max":{"1min":16.489999999999998,"5min":18.992999999999999,"15min":18.992999999999999},"last":9.9149999999999991},{"interface":"front","average":{"1min":13.263,"5min":29.742999999999999,"15min":32.490000000000002},"min":{"1min":8.984,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":19.466999999999999,"5min":387.15600000000001,"15min":387.15600000000001},"last":13.932}]},{"osd":2,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.083,"5min":18.381,"15min":19.431000000000001},"min":{"1min":6.5800000000000001,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":31.495999999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":14.241,"5min":19.315000000000001,"15min":20.161000000000001},"min":{"1min":7.7249999999999996,"5min":7.7249999999999996,"15min":7.7249999999999996},"max":{"1min":20.09,"5min":143.578,"15min":143.578},"last":15.561}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.124000000000001,"5min":20.594999999999999,"15min":21.507000000000001},"min":{"1min":9.6370000000000005,"5min":9.6370000000000005,"15min":9.6370000000000005},"max":{"1min":21.492999999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":16.382000000000001},{"interface":"front","average":{"1min":18.352,"5min":24.007000000000001,"15min":24.949000000000002},"min":{"1min":13.48,"5min":13.48,"15min":13.48},"max":{"1min":30.067,"5min":161.32900000000001,"15min":161.32900000000001},"last":17.221}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":11.082000000000001,"5min":14.252000000000001,"15min":14.779999999999999},"min":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":18.454000000000001,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.1020000000000003},{"interface":"front","average":{"1min":13.582000000000001,"5min":15.278,"15min":15.56},"min":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"max":{"1min":20.756,"5min":58.613999999999997,"15min":58.613999999999997},"last":12.173}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":13.81,"5min":17.134,"15min":17.687999999999999},"min":{"1min":7.383,"5min":7.383,"15min":7.383},"max":{"1min":20.297000000000001,"5min":71.757000000000005,"15min":71.757000000000005},"last":8.9179999999999993},{"interface":"front","average":{"1min":17.872,"5min":20.984000000000002,"15min":21.501999999999999},"min":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"max":{"1min":24.132000000000001,"5min":80.119,"15min":80.119},"last":17.951000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.315000000000001,"5min":19.055,"15min":19.512},"min":{"1min":8.9610000000000003,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":29.869,"5min":81.247,"15min":81.247},"last":12.864000000000001},{"interface":"front","average":{"1min":16.158000000000001,"5min":18.501999999999999,"15min":18.891999999999999},"min":{"1min":10.763,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":27.731999999999999,"5min":70.989000000000004,"15min":70.989000000000004},"last":11.109}]}]},{"osd":3,"up_from":16,"seq":68719476773,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":10.17,"5min":10.882,"15min":11},"min":{"1min":6.6020000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":26.021000000000001,"5min":26.021000000000001,"15min":26.021000000000001},"last":12.115},{"interface":"front","average":{"1min":11.24,"5min":10.454000000000001,"15min":10.323},"min":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":27.683,"5min":27.683,"15min":27.683},"last":8.1539999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":12.688000000000001,"5min":23.998999999999999,"15min":25.884},"min":{"1min":7.4039999999999999,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":22.036000000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":18.942},{"interface":"front","average":{"1min":12.916,"5min":25.045999999999999,"15min":27.068000000000001},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24.646999999999998,"5min":190.34200000000001,"15min":190.34200000000001},"last":18.216000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.867000000000001,"5min":17.914999999999999,"15min":18.257000000000001},"min":{"1min":10.250999999999999,"5min":10.250999999999999,"15min":10.250999999999999},"max":{"1min":30.335000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":11.191000000000001},{"interface":"front","average":{"1min":14.234,"5min":16.606000000000002,"15min":17.001000000000001},"min":{"1min":9.1639999999999997,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":23.334,"5min":33.060000000000002,"15min":33.060000000000002},"last":14.647}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.425000000000001,"5min":19.581,"15min":20.273},"min":{"1min":10.442,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":23.326000000000001,"5min":42.274999999999999,"15min":42.274999999999999},"last":15.43},{"interface":"front","average":{"1min":9.7880000000000003,"5min":11.247999999999999,"15min":11.492000000000001},"min":{"1min":5.7969999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":19.716000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":13.555}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":18.329999999999998,"15min":19.216000000000001},"min":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"max":{"1min":22.614000000000001,"5min":56.796999999999997,"15min":56.796999999999997},"last":9.4770000000000003},{"interface":"front","average":{"1min":15.929,"5min":19.824999999999999,"15min":20.474},"min":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"max":{"1min":28.501000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":17.27}]},{"osd":6,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":19.527999999999999,"5min":27.216000000000001,"15min":28.497},"min":{"1min":10.875,"5min":10.875,"15min":10.875},"max":{"1min":41.771000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":16.042999999999999},{"interface":"front","average":{"1min":14.746,"5min":15.343999999999999,"15min":15.444000000000001},"min":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"max":{"1min":40.128999999999998,"5min":40.128999999999998,"15min":40.128999999999998},"last":6.5510000000000002}]},{"osd":7,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":16.882999999999999,"5min":17.765000000000001,"15min":17.911999999999999},"min":{"1min":10.404,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":31.125,"5min":38.887999999999998,"15min":38.887999999999998},"last":44.746000000000002},{"interface":"front","average":{"1min":19.367999999999999,"5min":27.565000000000001,"15min":28.931000000000001},"min":{"1min":12.272,"5min":10.025,"15min":10.025},"max":{"1min":29.981999999999999,"5min":170.232,"15min":170.232},"last":42.689999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.563000000000001,"5min":24.759,"15min":26.792000000000002},"min":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"max":{"1min":51.853000000000002,"5min":258.66199999999998,"15min":258.66199999999998},"last":9.8059999999999992},{"interface":"front","average":{"1min":12.401,"5min":21.513999999999999,"15min":23.033000000000001},"min":{"1min":6.6879999999999997,"5min":6.016,"15min":6.016},"max":{"1min":49.905000000000001,"5min":203.38499999999999,"15min":203.38499999999999},"last":12.289999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.252000000000001,"5min":15.499000000000001,"15min":16.039999999999999},"min":{"1min":6.6790000000000003,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":20.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":22.904},{"interface":"front","average":{"1min":11.571999999999999,"5min":14.185,"15min":14.621},"min":{"1min":8.8179999999999996,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":16.605,"5min":37.390999999999998,"15min":37.390999999999998},"last":15.984999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.577999999999999,"5min":17.576000000000001,"15min":18.742000000000001},"min":{"1min":6.9020000000000001,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":14.788,"5min":183.38900000000001,"15min":183.38900000000001},"last":8.0210000000000008},{"interface":"front","average":{"1min":14.112,"5min":21.276,"15min":22.469999999999999},"min":{"1min":8.2420000000000009,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.007000000000001,"5min":189.374,"15min":189.374},"last":24.317}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.077999999999999,"5min":20.582000000000001,"15min":21.5},"min":{"1min":11.079000000000001,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":27.879999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":13.766999999999999},{"interface":"front","average":{"1min":12.926,"5min":18.677,"15min":19.635000000000002},"min":{"1min":7.3929999999999998,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":26.975999999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":11.717000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.343,"5min":25.315999999999999,"15min":26.812000000000001},"min":{"1min":10.385,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":21.745000000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":19.259},{"interface":"front","average":{"1min":15.525,"5min":24.969000000000001,"15min":26.542999999999999},"min":{"1min":10.387,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":19.696999999999999,"5min":187.863,"15min":187.863},"last":18.067}]},{"osd":6,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.364000000000001,"5min":24.882000000000001,"15min":26.468},"min":{"1min":11.069000000000001,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":20.021999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":14.771000000000001},{"interface":"front","average":{"1min":18.702000000000002,"5min":28.018999999999998,"15min":29.571999999999999},"min":{"1min":11.772,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":25.587,"5min":275.92200000000003,"15min":275.92200000000003},"last":28.056999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":17.663,"5min":25.314,"15min":26.588999999999999},"min":{"1min":13.518000000000001,"5min":13.518000000000001,"15min":13.518000000000001},"max":{"1min":24.042999999999999,"5min":170.43600000000001,"15min":170.43600000000001},"last":25.423999999999999},{"interface":"front","average":{"1min":17.009,"5min":22.231999999999999,"15min":23.102},"min":{"1min":10.459,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":24.635000000000002,"5min":157.00200000000001,"15min":157.00200000000001},"last":20.588999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.038,"5min":10.962999999999999,"15min":10.784000000000001},"min":{"1min":4.5149999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":39.700000000000003,"5min":39.700000000000003,"15min":39.700000000000003},"last":9.1899999999999995},{"interface":"front","average":{"1min":11.220000000000001,"5min":11.77,"15min":11.862},"min":{"1min":5.2050000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":28.358000000000001,"5min":30.788,"15min":30.788},"last":8.2899999999999991}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.443,"5min":18.129999999999999,"15min":19.411000000000001},"min":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":20.760999999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":7.1769999999999996},{"interface":"front","average":{"1min":12.442,"5min":14.385999999999999,"15min":14.710000000000001},"min":{"1min":6.8070000000000004,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":23.695,"5min":35.834000000000003,"15min":35.834000000000003},"last":11.340999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.311,"5min":20.981999999999999,"15min":22.094000000000001},"min":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":34.953000000000003,"5min":99.790999999999997,"15min":99.790999999999997},"last":10.712},{"interface":"front","average":{"1min":14.433,"5min":16.257999999999999,"15min":16.562000000000001},"min":{"1min":9.5980000000000008,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":33.680999999999997,"5min":33.680999999999997,"15min":33.680999999999997},"last":18.760000000000002}]},{"osd":4,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.423999999999999,"5min":15.603999999999999,"15min":16.300999999999998},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":19.856999999999999,"5min":41.134999999999998,"15min":41.134999999999998},"last":10.109},{"interface":"front","average":{"1min":15.153,"5min":23.329000000000001,"15min":24.692},"min":{"1min":8.5579999999999998,"5min":8.5579999999999998,"15min":8.5579999999999998},"max":{"1min":23.863,"5min":103.705,"15min":103.705},"last":17.050000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.125,"5min":18.056000000000001,"15min":18.710999999999999},"min":{"1min":8.4629999999999992,"5min":8.4629999999999992,"15min":8.4629999999999992},"max":{"1min":26.533999999999999,"5min":43.847000000000001,"15min":43.847000000000001},"last":13.653},{"interface":"front","average":{"1min":16.975999999999999,"5min":26.073,"15min":27.588999999999999},"min":{"1min":9.6270000000000007,"5min":9.6270000000000007,"15min":9.6270000000000007},"max":{"1min":28.102,"5min":98.707999999999998,"15min":98.707999999999998},"last":12.885}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":18.401,"5min":20.082999999999998,"15min":20.363},"min":{"1min":8.7550000000000008,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":55.631,"5min":55.631,"15min":55.631},"last":15.637},{"interface":"front","average":{"1min":17.962,"5min":21.433,"15min":22.010999999999999},"min":{"1min":10.673999999999999,"5min":10.673999999999999,"15min":10.673999999999999},"max":{"1min":54.656999999999996,"5min":54.656999999999996,"15min":54.656999999999996},"last":15.012}]},{"osd":7,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":17.059000000000001,"5min":21.111000000000001,"15min":21.786000000000001},"min":{"1min":12.289,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":23.265000000000001,"5min":45.927999999999997,"15min":45.927999999999997},"last":19.673999999999999},{"interface":"front","average":{"1min":17.498999999999999,"5min":22.126000000000001,"15min":22.896999999999998},"min":{"1min":11.305999999999999,"5min":11.305999999999999,"15min":11.305999999999999},"max":{"1min":28.651,"5min":45.130000000000003,"15min":45.130000000000003},"last":16.356999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":9.0389999999999997,"5min":14.882999999999999,"15min":15.856999999999999},"min":{"1min":4.431,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":17.584,"5min":131.477,"15min":131.477},"last":6.5090000000000003},{"interface":"front","average":{"1min":9.7940000000000005,"5min":15.146000000000001,"15min":16.038},"min":{"1min":6.0839999999999996,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":20.175999999999998,"5min":129.386,"15min":129.386},"last":7.7370000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":10.300000000000001,"5min":10.855,"15min":10.946999999999999},"min":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"max":{"1min":14.603,"5min":25.774000000000001,"15min":25.774000000000001},"last":13.699},{"interface":"front","average":{"1min":16.196000000000002,"5min":18.829999999999998,"15min":19.268999999999998},"min":{"1min":11.680999999999999,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":27.390999999999998,"5min":91.295000000000002,"15min":91.295000000000002},"last":15.038}]},{"osd":2,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.880000000000001,"5min":17.151,"15min":17.863},"min":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":46.811,"5min":134.21199999999999,"15min":134.21199999999999},"last":22.661999999999999},{"interface":"front","average":{"1min":14.634,"5min":14.634,"15min":14.634},"min":{"1min":5.992,"5min":5.992,"15min":5.992},"max":{"1min":50.450000000000003,"5min":50.450000000000003,"15min":50.450000000000003},"last":18.574000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.888,"5min":19.111999999999998,"15min":20.149999999999999},"min":{"1min":6.758,"5min":6.758,"15min":6.758},"max":{"1min":22.356999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":17.547999999999998},{"interface":"front","average":{"1min":15.807,"5min":20.111000000000001,"15min":20.829000000000001},"min":{"1min":9.1150000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":25.867000000000001,"5min":124.684,"15min":124.684},"last":14.35}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":12.601000000000001,"5min":13.630000000000001,"15min":13.802},"min":{"1min":8.5579999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":17.361999999999998,"5min":24.963000000000001,"15min":24.963000000000001},"last":20.908000000000001},{"interface":"front","average":{"1min":16.216000000000001,"5min":15.724,"15min":15.641999999999999},"min":{"1min":11.066000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":26.544,"5min":26.544,"15min":26.544},"last":25.838999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.038,"5min":19.187000000000001,"15min":20.212},"min":{"1min":7.016,"5min":7.016,"15min":7.016},"max":{"1min":18.068999999999999,"5min":125.792,"15min":125.792},"last":26.137},{"interface":"front","average":{"1min":16.5,"5min":16.905000000000001,"15min":16.972999999999999},"min":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"max":{"1min":24.768000000000001,"5min":28.591999999999999,"15min":28.591999999999999},"last":21.574999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":16.189,"5min":28.972000000000001,"15min":31.102},"min":{"1min":7.6020000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":24.219000000000001,"5min":327.29599999999999,"15min":327.29599999999999},"last":15.619},{"interface":"front","average":{"1min":14.366,"5min":30.007999999999999,"15min":32.615000000000002},"min":{"1min":7.5499999999999998,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":18.904,"5min":323.82400000000001,"15min":323.82400000000001},"last":20.329999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":645744,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.606999999999999,"5min":14.191000000000001,"15min":14.788},"min":{"1min":5.5609999999999999,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":28.821999999999999,"5min":68.513999999999996,"15min":68.513999999999996},"last":11.003},{"interface":"front","average":{"1min":10.786,"5min":15.789999999999999,"15min":16.623999999999999},"min":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":27.027999999999999,"5min":88.816000000000003,"15min":88.816000000000003},"last":6.4530000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.936999999999999,"5min":16.495000000000001,"15min":17.254999999999999},"min":{"1min":7.3710000000000004,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":19.015999999999998,"5min":87.515000000000001,"15min":87.515000000000001},"last":30.579999999999998},{"interface":"front","average":{"1min":12.638999999999999,"5min":16.937999999999999,"15min":17.654},"min":{"1min":6.5549999999999997,"5min":6.21,"15min":6.21},"max":{"1min":26.303000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":28.934999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15.009,"5min":17.692,"15min":18.138999999999999},"min":{"1min":10.311,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":23.599,"5min":113.943,"15min":113.943},"last":25.640000000000001},{"interface":"front","average":{"1min":11.092000000000001,"5min":14.593999999999999,"15min":15.178000000000001},"min":{"1min":5.7949999999999999,"5min":5.7949999999999999,"15min":5.7949999999999999},"max":{"1min":22.634,"5min":80.004000000000005,"15min":80.004000000000005},"last":33.826000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":16.504999999999999,"5min":20.773,"15min":21.484999999999999},"min":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":87.866,"5min":87.866,"15min":87.866},"last":8.0009999999999994},{"interface":"front","average":{"1min":19.504999999999999,"5min":23.062000000000001,"15min":23.655000000000001},"min":{"1min":11.359999999999999,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":89.334000000000003,"5min":91.183000000000007,"15min":91.183000000000007},"last":11.882}]},{"osd":4,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":12.736000000000001,"5min":12.737,"15min":12.737},"min":{"1min":7.9050000000000002,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":20.556999999999999,"5min":25.195,"15min":25.195},"last":14.602},{"interface":"front","average":{"1min":14.071,"5min":16.562999999999999,"15min":16.978000000000002},"min":{"1min":7.2279999999999998,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":19.228000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":23.651}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.146000000000001,"5min":15.726000000000001,"15min":15.99},"min":{"1min":7.6980000000000004,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":46.561999999999998,"5min":66.765000000000001,"15min":66.765000000000001},"last":9.2989999999999995},{"interface":"front","average":{"1min":17.895,"5min":20.585000000000001,"15min":21.033000000000001},"min":{"1min":11.125999999999999,"5min":11.125999999999999,"15min":11.125999999999999},"max":{"1min":39.637999999999998,"5min":71.771000000000001,"15min":71.771000000000001},"last":11.585000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":17.015000000000001,"5min":18.721,"15min":19.004999999999999},"min":{"1min":10.923,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":25.045999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":15.161},{"interface":"front","average":{"1min":15.677,"5min":17.84,"15min":18.199999999999999},"min":{"1min":8.1170000000000009,"5min":8.1170000000000009,"15min":8.1170000000000009},"max":{"1min":21.984000000000002,"5min":42.531999999999996,"15min":42.531999999999996},"last":29.597999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":8.6489999999999991,"5min":9.1769999999999996,"15min":9.2650000000000006},"min":{"1min":4.6660000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":26.984999999999999,"5min":26.984999999999999,"15min":26.984999999999999},"last":8.0329999999999995},{"interface":"front","average":{"1min":11.923999999999999,"5min":10.831,"15min":10.648999999999999},"min":{"1min":6.4500000000000002,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":25.934999999999999,"5min":25.934999999999999,"15min":25.934999999999999},"last":10.390000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:20 2023","interfaces":[{"interface":"back","average":{"1min":12.436999999999999,"5min":11.664,"15min":11.535},"min":{"1min":6.9859999999999998,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":6.8680000000000003},{"interface":"front","average":{"1min":15.724,"5min":14.467000000000001,"15min":14.257},"min":{"1min":9.3049999999999997,"5min":6.609,"15min":6.609},"max":{"1min":30.751999999999999,"5min":30.751999999999999,"15min":30.751999999999999},"last":9.0220000000000002}]},{"osd":2,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":14.199,"5min":12.414,"15min":12.116},"min":{"1min":7.4530000000000003,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":24.254999999999999,"5min":24.254999999999999,"15min":24.254999999999999},"last":13.795999999999999},{"interface":"front","average":{"1min":15.834,"5min":15.787000000000001,"15min":15.779},"min":{"1min":9.625,"5min":9.625,"15min":9.625},"max":{"1min":25.428000000000001,"5min":25.428000000000001,"15min":25.428000000000001},"last":12.448}]},{"osd":3,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":10.042,"5min":11.02,"15min":11.183},"min":{"1min":6.415,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":32.927999999999997,"5min":32.927999999999997,"15min":32.927999999999997},"last":9.6449999999999996},{"interface":"front","average":{"1min":14.454000000000001,"5min":15.743,"15min":15.958},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":19.853000000000002,"5min":30.285,"15min":30.285},"last":16.638000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.544,"5min":19.591000000000001,"15min":19.599},"min":{"1min":10.619999999999999,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":40.25,"5min":64.474999999999994,"15min":64.474999999999994},"last":13.048999999999999},{"interface":"front","average":{"1min":15.550000000000001,"5min":16.079000000000001,"15min":16.167000000000002},"min":{"1min":6.9809999999999999,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":37.826999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":17.988}]},{"osd":6,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":20.481999999999999,"5min":19.119,"15min":18.891999999999999},"min":{"1min":14.273,"5min":14.273,"15min":14.273},"max":{"1min":35.241,"5min":35.241,"15min":35.241},"last":11.004},{"interface":"front","average":{"1min":14.154,"5min":14.051,"15min":14.034000000000001},"min":{"1min":8.8379999999999992,"5min":7.343,"15min":7.343},"max":{"1min":38.668999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":17.379999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":16.852,"5min":18.056000000000001,"15min":18.257000000000001},"min":{"1min":10.457000000000001,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":41.783999999999999,"5min":74.977999999999994,"15min":74.977999999999994},"last":14.423},{"interface":"front","average":{"1min":21.052,"5min":22.094000000000001,"15min":22.268000000000001},"min":{"1min":15.183999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":44.450000000000003,"5min":83.138000000000005,"15min":83.138000000000005},"last":11.701000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:26:53.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:26:54.023 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:54.022+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:54.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:54.046 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:54.100 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.100 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.107 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:54.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:54.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:54.172 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:54.172 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:26:54.208 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:54.206+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:54.280 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.280 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:54.290 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:54.291 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:26:54.309 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:54.308+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:54.349 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:24.353 96291== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:54.398 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:54.399 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:26:54.410 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:54.409+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:54.430 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.430 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.436 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:24.440 96291== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:54.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:54.512 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:54.513 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:26:54.513 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:54.512+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:54.523 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:26:54.612 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:54.611+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:54.622 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.623 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:54.695 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:26:54.711 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2023-10-30T06:26:54.506689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221060,"kb_used_data":4888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753660,"statfs":{"total":773094113280,"available":772867747840,"internally_reserved":0,"allocated":5005312,"data_stored":2801060,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.222648"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.295296+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:26:50.295296+0000","last_peered":"2023-10-30T06:26:50.295296+0000","last_clean":"2023-10-30T06:26:50.295296+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:26:50.295296+0000","last_undegraded":"2023-10-30T06:26:50.295296+0000","last_fullsized":"2023-10-30T06:26:50.295296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.980342+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:26:48.980342+0000","last_peered":"2023-10-30T06:26:48.980342+0000","last_clean":"2023-10-30T06:26:48.980342+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:26:48.980342+0000","last_undegraded":"2023-10-30T06:26:48.980342+0000","last_fullsized":"2023-10-30T06:26:48.980342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-30T06:26:52.293348+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:26:52.293348+0000","last_peered":"2023-10-30T06:26:52.293348+0000","last_clean":"2023-10-30T06:26:52.293348+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:26:52.293348+0000","last_undegraded":"2023-10-30T06:26:52.293348+0000","last_fullsized":"2023-10-30T06:26:52.293348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.899988+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:26:48.899988+0000","last_peered":"2023-10-30T06:26:48.899988+0000","last_clean":"2023-10-30T06:26:48.899988+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:26:48.899988+0000","last_undegraded":"2023-10-30T06:26:48.899988+0000","last_fullsized":"2023-10-30T06:26:48.899988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.290952+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:26:50.290952+0000","last_peered":"2023-10-30T06:26:50.290952+0000","last_clean":"2023-10-30T06:26:50.290952+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:26:50.290952+0000","last_undegraded":"2023-10-30T06:26:50.290952+0000","last_fullsized":"2023-10-30T06:26:50.290952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.393598+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:26:49.393598+0000","last_peered":"2023-10-30T06:26:49.393598+0000","last_clean":"2023-10-30T06:26:49.393598+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:26:49.393598+0000","last_undegraded":"2023-10-30T06:26:49.393598+0000","last_fullsized":"2023-10-30T06:26:49.393598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.286753+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:26:50.286753+0000","last_peered":"2023-10-30T06:26:50.286753+0000","last_clean":"2023-10-30T06:26:50.286753+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:26:50.286753+0000","last_undegraded":"2023-10-30T06:26:50.286753+0000","last_fullsized":"2023-10-30T06:26:50.286753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.395336+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:26:49.395336+0000","last_peered":"2023-10-30T06:26:49.395336+0000","last_clean":"2023-10-30T06:26:49.395336+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:26:49.395336+0000","last_undegraded":"2023-10-30T06:26:49.395336+0000","last_fullsized":"2023-10-30T06:26:49.395336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":150,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-30T06:26:52.295628+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:26:52.295628+0000","last_peered":"2023-10-30T06:26:52.295628+0000","last_clean":"2023-10-30T06:26:52.295628+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:26:52.295628+0000","last_undegraded":"2023-10-30T06:26:52.295628+0000","last_fullsized":"2023-10-30T06:26:52.295628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411364,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":656818,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":11.029999999999999,"5min":10.494,"15min":10.404},"min":{"1min":5.2720000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"last":15.856999999999999},{"interface":"front","average":{"1min":12.122999999999999,"5min":12.090999999999999,"15min":12.085000000000001},"min":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":35.398000000000003,"5min":46.798000000000002,"15min":46.798000000000002},"last":7.4059999999999997}]},{"osd":2,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":13.641,"15min":13.744999999999999},"min":{"1min":5.7350000000000003,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":44.537999999999997,"5min":52.552,"15min":52.552},"last":14.476000000000001},{"interface":"front","average":{"1min":17.143000000000001,"5min":15.545999999999999,"15min":15.279999999999999},"min":{"1min":7.391,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":55.585999999999999,"5min":55.585999999999999,"15min":55.585999999999999},"last":9.2059999999999995}]},{"osd":3,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":15.367000000000001,"5min":21.626000000000001,"15min":22.669},"min":{"1min":8.5980000000000008,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":32.259999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":15.214},{"interface":"front","average":{"1min":14.832000000000001,"5min":21.024000000000001,"15min":22.056000000000001},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":38.987000000000002,"5min":124.075,"15min":124.075},"last":10.411}]},{"osd":4,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":13.747,"15min":13.699},"min":{"1min":7.6470000000000002,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":25.023,"5min":51.649000000000001,"15min":51.649000000000001},"last":8.5649999999999995},{"interface":"front","average":{"1min":18.245000000000001,"5min":18.047999999999998,"15min":18.015000000000001},"min":{"1min":10.759,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":44.973999999999997,"5min":44.973999999999997,"15min":44.973999999999997},"last":17.225999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":16.564,"5min":20.154,"15min":20.751999999999999},"min":{"1min":7.351,"5min":7.351,"15min":7.351},"max":{"1min":58.648000000000003,"5min":123.754,"15min":123.754},"last":11.435},{"interface":"front","average":{"1min":14.667999999999999,"5min":21.396999999999998,"15min":22.518999999999998},"min":{"1min":7.6619999999999999,"5min":7.6619999999999999,"15min":7.6619999999999999},"max":{"1min":41.558999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":13.851000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":17.972000000000001,"5min":20.760000000000002,"15min":21.224},"min":{"1min":9.359,"5min":9.359,"15min":9.359},"max":{"1min":39.860999999999997,"5min":66.400999999999996,"15min":66.400999999999996},"last":12.01},{"interface":"front","average":{"1min":16.838999999999999,"5min":19.254999999999999,"15min":19.657},"min":{"1min":9.6969999999999992,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":37.027999999999999,"5min":53.149000000000001,"15min":53.149000000000001},"last":12.611000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":18.468,"5min":20.904,"15min":21.309999999999999},"min":{"1min":10.083,"5min":10.083,"15min":10.083},"max":{"1min":51.375,"5min":51.375,"15min":51.375},"last":18.904},{"interface":"front","average":{"1min":19.795999999999999,"5min":21.234999999999999,"15min":21.475000000000001},"min":{"1min":10.004,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":54.439,"5min":69.991,"15min":69.991},"last":16.504000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476773,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.561,"5min":13.877000000000001,"15min":14.429},"min":{"1min":4.9980000000000002,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":37.701999999999998,"5min":94.343000000000004,"15min":94.343000000000004},"last":14.741},{"interface":"front","average":{"1min":11.94,"5min":14.991,"15min":15.499000000000001},"min":{"1min":5.8700000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":49.045999999999999,"5min":122.054,"15min":122.054},"last":5.6390000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.071,"5min":10.667,"15min":10.766},"min":{"1min":5.8789999999999996,"5min":5.8789999999999996,"15min":5.8789999999999996},"max":{"1min":16.489999999999998,"5min":18.992999999999999,"15min":18.992999999999999},"last":9.9149999999999991},{"interface":"front","average":{"1min":13.263,"5min":29.742999999999999,"15min":32.490000000000002},"min":{"1min":8.984,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":19.466999999999999,"5min":387.15600000000001,"15min":387.15600000000001},"last":13.932}]},{"osd":2,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.083,"5min":18.381,"15min":19.431000000000001},"min":{"1min":6.5800000000000001,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":31.495999999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":14.241,"5min":19.315000000000001,"15min":20.161000000000001},"min":{"1min":7.7249999999999996,"5min":7.7249999999999996,"15min":7.7249999999999996},"max":{"1min":20.09,"5min":143.578,"15min":143.578},"last":15.561}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.124000000000001,"5min":20.594999999999999,"15min":21.507000000000001},"min":{"1min":9.6370000000000005,"5min":9.6370000000000005,"15min":9.6370000000000005},"max":{"1min":21.492999999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":16.382000000000001},{"interface":"front","average":{"1min":18.352,"5min":24.007000000000001,"15min":24.949000000000002},"min":{"1min":13.48,"5min":13.48,"15min":13.48},"max":{"1min":30.067,"5min":161.32900000000001,"15min":161.32900000000001},"last":17.221}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":11.082000000000001,"5min":14.252000000000001,"15min":14.779999999999999},"min":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":18.454000000000001,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.1020000000000003},{"interface":"front","average":{"1min":13.582000000000001,"5min":15.278,"15min":15.56},"min":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"max":{"1min":20.756,"5min":58.613999999999997,"15min":58.613999999999997},"last":12.173}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":13.81,"5min":17.134,"15min":17.687999999999999},"min":{"1min":7.383,"5min":7.383,"15min":7.383},"max":{"1min":20.297000000000001,"5min":71.757000000000005,"15min":71.757000000000005},"last":8.9179999999999993},{"interface":"front","average":{"1min":17.872,"5min":20.984000000000002,"15min":21.501999999999999},"min":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"max":{"1min":24.132000000000001,"5min":80.119,"15min":80.119},"last":17.951000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.315000000000001,"5min":19.055,"15min":19.512},"min":{"1min":8.9610000000000003,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":29.869,"5min":81.247,"15min":81.247},"last":12.864000000000001},{"interface":"front","average":{"1min":16.158000000000001,"5min":18.501999999999999,"15min":18.891999999999999},"min":{"1min":10.763,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":27.731999999999999,"5min":70.989000000000004,"15min":70.989000000000004},"last":11.109}]}]},{"osd":3,"up_from":16,"seq":68719476773,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":10.17,"5min":10.882,"15min":11},"min":{"1min":6.6020000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":26.021000000000001,"5min":26.021000000000001,"15min":26.021000000000001},"last":12.115},{"interface":"front","average":{"1min":11.24,"5min":10.454000000000001,"15min":10.323},"min":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":27.683,"5min":27.683,"15min":27.683},"last":8.1539999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":12.688000000000001,"5min":23.998999999999999,"15min":25.884},"min":{"1min":7.4039999999999999,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":22.036000000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":18.942},{"interface":"front","average":{"1min":12.916,"5min":25.045999999999999,"15min":27.068000000000001},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24.646999999999998,"5min":190.34200000000001,"15min":190.34200000000001},"last":18.216000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.867000000000001,"5min":17.914999999999999,"15min":18.257000000000001},"min":{"1min":10.250999999999999,"5min":10.250999999999999,"15min":10.250999999999999},"max":{"1min":30.335000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":11.191000000000001},{"interface":"front","average":{"1min":14.234,"5min":16.606000000000002,"15min":17.001000000000001},"min":{"1min":9.1639999999999997,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":23.334,"5min":33.060000000000002,"15min":33.060000000000002},"last":14.647}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.425000000000001,"5min":19.581,"15min":20.273},"min":{"1min":10.442,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":23.326000000000001,"5min":42.274999999999999,"15min":42.274999999999999},"last":15.43},{"interface":"front","average":{"1min":9.7880000000000003,"5min":11.247999999999999,"15min":11.492000000000001},"min":{"1min":5.7969999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":19.716000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":13.555}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":18.329999999999998,"15min":19.216000000000001},"min":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"max":{"1min":22.614000000000001,"5min":56.796999999999997,"15min":56.796999999999997},"last":9.4770000000000003},{"interface":"front","average":{"1min":15.929,"5min":19.824999999999999,"15min":20.474},"min":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"max":{"1min":28.501000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":17.27}]},{"osd":6,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":19.527999999999999,"5min":27.216000000000001,"15min":28.497},"min":{"1min":10.875,"5min":10.875,"15min":10.875},"max":{"1min":41.771000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":16.042999999999999},{"interface":"front","average":{"1min":14.746,"5min":15.343999999999999,"15min":15.444000000000001},"min":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"max":{"1min":40.128999999999998,"5min":40.128999999999998,"15min":40.128999999999998},"last":6.5510000000000002}]},{"osd":7,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":16.882999999999999,"5min":17.765000000000001,"15min":17.911999999999999},"min":{"1min":10.404,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":31.125,"5min":38.887999999999998,"15min":38.887999999999998},"last":44.746000000000002},{"interface":"front","average":{"1min":19.367999999999999,"5min":27.565000000000001,"15min":28.931000000000001},"min":{"1min":12.272,"5min":10.025,"15min":10.025},"max":{"1min":29.981999999999999,"5min":170.232,"15min":170.232},"last":42.689999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.563000000000001,"5min":24.759,"15min":26.792000000000002},"min":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"max":{"1min":51.853000000000002,"5min":258.66199999999998,"15min":258.66199999999998},"last":9.8059999999999992},{"interface":"front","average":{"1min":12.401,"5min":21.513999999999999,"15min":23.033000000000001},"min":{"1min":6.6879999999999997,"5min":6.016,"15min":6.016},"max":{"1min":49.905000000000001,"5min":203.38499999999999,"15min":203.38499999999999},"last":12.289999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.252000000000001,"5min":15.499000000000001,"15min":16.039999999999999},"min":{"1min":6.6790000000000003,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":20.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":22.904},{"interface":"front","average":{"1min":11.571999999999999,"5min":14.185,"15min":14.621},"min":{"1min":8.8179999999999996,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":16.605,"5min":37.390999999999998,"15min":37.390999999999998},"last":15.984999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.577999999999999,"5min":17.576000000000001,"15min":18.742000000000001},"min":{"1min":6.9020000000000001,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":14.788,"5min":183.38900000000001,"15min":183.38900000000001},"last":8.0210000000000008},{"interface":"front","average":{"1min":14.112,"5min":21.276,"15min":22.469999999999999},"min":{"1min":8.2420000000000009,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.007000000000001,"5min":189.374,"15min":189.374},"last":24.317}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.077999999999999,"5min":20.582000000000001,"15min":21.5},"min":{"1min":11.079000000000001,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":27.879999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":13.766999999999999},{"interface":"front","average":{"1min":12.926,"5min":18.677,"15min":19.635000000000002},"min":{"1min":7.3929999999999998,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":26.975999999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":11.717000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.343,"5min":25.315999999999999,"15min":26.812000000000001},"min":{"1min":10.385,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":21.745000000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":19.259},{"interface":"front","average":{"1min":15.525,"5min":24.969000000000001,"15min":26.542999999999999},"min":{"1min":10.387,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":19.696999999999999,"5min":187.863,"15min":187.863},"last":18.067}]},{"osd":6,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.364000000000001,"5min":24.882000000000001,"15min":26.468},"min":{"1min":11.069000000000001,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":20.021999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":14.771000000000001},{"interface":"front","average":{"1min":18.702000000000002,"5min":28.018999999999998,"15min":29.571999999999999},"min":{"1min":11.772,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":25.587,"5min":275.92200000000003,"15min":275.92200000000003},"last":28.056999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":17.663,"5min":25.314,"15min":26.588999999999999},"min":{"1min":13.518000000000001,"5min":13.518000000000001,"15min":13.518000000000001},"max":{"1min":24.042999999999999,"5min":170.43600000000001,"15min":170.43600000000001},"last":25.423999999999999},{"interface":"front","average":{"1min":17.009,"5min":22.231999999999999,"15min":23.102},"min":{"1min":10.459,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":24.635000000000002,"5min":157.00200000000001,"15min":157.00200000000001},"last":20.588999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.038,"5min":10.962999999999999,"15min":10.784000000000001},"min":{"1min":4.5149999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":39.700000000000003,"5min":39.700000000000003,"15min":39.700000000000003},"last":9.1899999999999995},{"interface":"front","average":{"1min":11.220000000000001,"5min":11.77,"15min":11.862},"min":{"1min":5.2050000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":28.358000000000001,"5min":30.788,"15min":30.788},"last":8.2899999999999991}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.443,"5min":18.129999999999999,"15min":19.411000000000001},"min":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":20.760999999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":7.1769999999999996},{"interface":"front","average":{"1min":12.442,"5min":14.385999999999999,"15min":14.710000000000001},"min":{"1min":6.8070000000000004,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":23.695,"5min":35.834000000000003,"15min":35.834000000000003},"last":11.340999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.311,"5min":20.981999999999999,"15min":22.094000000000001},"min":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":34.953000000000003,"5min":99.790999999999997,"15min":99.790999999999997},"last":10.712},{"interface":"front","average":{"1min":14.433,"5min":16.257999999999999,"15min":16.562000000000001},"min":{"1min":9.5980000000000008,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":33.680999999999997,"5min":33.680999999999997,"15min":33.680999999999997},"last":18.760000000000002}]},{"osd":4,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.423999999999999,"5min":15.603999999999999,"15min":16.300999999999998},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":19.856999999999999,"5min":41.134999999999998,"15min":41.134999999999998},"last":10.109},{"interface":"front","average":{"1min":15.153,"5min":23.329000000000001,"15min":24.692},"min":{"1min":8.5579999999999998,"5min":8.5579999999999998,"15min":8.5579999999999998},"max":{"1min":23.863,"5min":103.705,"15min":103.705},"last":17.050000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.125,"5min":18.056000000000001,"15min":18.710999999999999},"min":{"1min":8.4629999999999992,"5min":8.4629999999999992,"15min":8.4629999999999992},"max":{"1min":26.533999999999999,"5min":43.847000000000001,"15min":43.847000000000001},"last":13.653},{"interface":"front","average":{"1min":16.975999999999999,"5min":26.073,"15min":27.588999999999999},"min":{"1min":9.6270000000000007,"5min":9.6270000000000007,"15min":9.6270000000000007},"max":{"1min":28.102,"5min":98.707999999999998,"15min":98.707999999999998},"last":12.885}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":18.401,"5min":20.082999999999998,"15min":20.363},"min":{"1min":8.7550000000000008,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":55.631,"5min":55.631,"15min":55.631},"last":15.637},{"interface":"front","average":{"1min":17.962,"5min":21.433,"15min":22.010999999999999},"min":{"1min":10.673999999999999,"5min":10.673999999999999,"15min":10.673999999999999},"max":{"1min":54.656999999999996,"5min":54.656999999999996,"15min":54.656999999999996},"last":15.012}]},{"osd":7,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":17.059000000000001,"5min":21.111000000000001,"15min":21.786000000000001},"min":{"1min":12.289,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":23.265000000000001,"5min":45.927999999999997,"15min":45.927999999999997},"last":19.673999999999999},{"interface":"front","average":{"1min":17.498999999999999,"5min":22.126000000000001,"15min":22.896999999999998},"min":{"1min":11.305999999999999,"5min":11.305999999999999,"15min":11.305999999999999},"max":{"1min":28.651,"5min":45.130000000000003,"15min":45.130000000000003},"last":16.356999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":9.0389999999999997,"5min":14.882999999999999,"15min":15.856999999999999},"min":{"1min":4.431,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":17.584,"5min":131.477,"15min":131.477},"last":6.5090000000000003},{"interface":"front","average":{"1min":9.7940000000000005,"5min":15.146000000000001,"15min":16.038},"min":{"1min":6.0839999999999996,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":20.175999999999998,"5min":129.386,"15min":129.386},"last":7.7370000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":10.300000000000001,"5min":10.855,"15min":10.946999999999999},"min":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"max":{"1min":14.603,"5min":25.774000000000001,"15min":25.774000000000001},"last":13.699},{"interface":"front","average":{"1min":16.196000000000002,"5min":18.829999999999998,"15min":19.268999999999998},"min":{"1min":11.680999999999999,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":27.390999999999998,"5min":91.295000000000002,"15min":91.295000000000002},"last":15.038}]},{"osd":2,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.880000000000001,"5min":17.151,"15min":17.863},"min":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":46.811,"5min":134.21199999999999,"15min":134.21199999999999},"last":22.661999999999999},{"interface":"front","average":{"1min":14.634,"5min":14.634,"15min":14.634},"min":{"1min":5.992,"5min":5.992,"15min":5.992},"max":{"1min":50.450000000000003,"5min":50.450000000000003,"15min":50.450000000000003},"last":18.574000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.888,"5min":19.111999999999998,"15min":20.149999999999999},"min":{"1min":6.758,"5min":6.758,"15min":6.758},"max":{"1min":22.356999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":17.547999999999998},{"interface":"front","average":{"1min":15.807,"5min":20.111000000000001,"15min":20.829000000000001},"min":{"1min":9.1150000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":25.867000000000001,"5min":124.684,"15min":124.684},"last":14.35}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":12.601000000000001,"5min":13.630000000000001,"15min":13.802},"min":{"1min":8.5579999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":17.361999999999998,"5min":24.963000000000001,"15min":24.963000000000001},"last":20.908000000000001},{"interface":"front","average":{"1min":16.216000000000001,"5min":15.724,"15min":15.641999999999999},"min":{"1min":11.066000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":26.544,"5min":26.544,"15min":26.544},"last":25.838999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.038,"5min":19.187000000000001,"15min":20.212},"min":{"1min":7.016,"5min":7.016,"15min":7.016},"max":{"1min":18.068999999999999,"5min":125.792,"15min":125.792},"last":26.137},{"interface":"front","average":{"1min":16.5,"5min":16.905000000000001,"15min":16.972999999999999},"min":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"max":{"1min":24.768000000000001,"5min":28.591999999999999,"15min":28.591999999999999},"last":21.574999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":16.189,"5min":28.972000000000001,"15min":31.102},"min":{"1min":7.6020000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":24.219000000000001,"5min":327.29599999999999,"15min":327.29599999999999},"last":15.619},{"interface":"front","average":{"1min":14.366,"5min":30.007999999999999,"15min":32.615000000000002},"min":{"1min":7.5499999999999998,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":18.904,"5min":323.82400000000001,"15min":323.82400000000001},"last":20.329999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":645744,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.606999999999999,"5min":14.191000000000001,"15min":14.788},"min":{"1min":5.5609999999999999,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":28.821999999999999,"5min":68.513999999999996,"15min":68.513999999999996},"last":11.003},{"interface":"front","average":{"1min":10.786,"5min":15.789999999999999,"15min":16.623999999999999},"min":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":27.027999999999999,"5min":88.816000000000003,"15min":88.816000000000003},"last":6.4530000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.936999999999999,"5min":16.495000000000001,"15min":17.254999999999999},"min":{"1min":7.3710000000000004,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":19.015999999999998,"5min":87.515000000000001,"15min":87.515000000000001},"last":30.579999999999998},{"interface":"front","average":{"1min":12.638999999999999,"5min":16.937999999999999,"15min":17.654},"min":{"1min":6.5549999999999997,"5min":6.21,"15min":6.21},"max":{"1min":26.303000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":28.934999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15.009,"5min":17.692,"15min":18.138999999999999},"min":{"1min":10.311,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":23.599,"5min":113.943,"15min":113.943},"last":25.640000000000001},{"interface":"front","average":{"1min":11.092000000000001,"5min":14.593999999999999,"15min":15.178000000000001},"min":{"1min":5.7949999999999999,"5min":5.7949999999999999,"15min":5.7949999999999999},"max":{"1min":22.634,"5min":80.004000000000005,"15min":80.004000000000005},"last":33.826000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":16.504999999999999,"5min":20.773,"15min":21.484999999999999},"min":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":87.866,"5min":87.866,"15min":87.866},"last":8.0009999999999994},{"interface":"front","average":{"1min":19.504999999999999,"5min":23.062000000000001,"15min":23.655000000000001},"min":{"1min":11.359999999999999,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":89.334000000000003,"5min":91.183000000000007,"15min":91.183000000000007},"last":11.882}]},{"osd":4,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":12.736000000000001,"5min":12.737,"15min":12.737},"min":{"1min":7.9050000000000002,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":20.556999999999999,"5min":25.195,"15min":25.195},"last":14.602},{"interface":"front","average":{"1min":14.071,"5min":16.562999999999999,"15min":16.978000000000002},"min":{"1min":7.2279999999999998,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":19.228000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":23.651}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.146000000000001,"5min":15.726000000000001,"15min":15.99},"min":{"1min":7.6980000000000004,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":46.561999999999998,"5min":66.765000000000001,"15min":66.765000000000001},"last":9.2989999999999995},{"interface":"front","average":{"1min":17.895,"5min":20.585000000000001,"15min":21.033000000000001},"min":{"1min":11.125999999999999,"5min":11.125999999999999,"15min":11.125999999999999},"max":{"1min":39.637999999999998,"5min":71.771000000000001,"15min":71.771000000000001},"last":11.585000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":17.015000000000001,"5min":18.721,"15min":19.004999999999999},"min":{"1min":10.923,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":25.045999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":15.161},{"interface":"front","average":{"1min":15.677,"5min":17.84,"15min":18.199999999999999},"min":{"1min":8.1170000000000009,"5min":8.1170000000000009,"15min":8.1170000000000009},"max":{"1min":21.984000000000002,"5min":42.531999999999996,"15min":42.531999999999996},"last":29.597999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":8.6489999999999991,"5min":9.1769999999999996,"15min":9.2650000000000006},"min":{"1min":4.6660000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":26.984999999999999,"5min":26.984999999999999,"15min":26.984999999999999},"last":8.0329999999999995},{"interface":"front","average":{"1min":11.923999999999999,"5min":10.831,"15min":10.648999999999999},"min":{"1min":6.4500000000000002,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":25.934999999999999,"5min":25.934999999999999,"15min":25.934999999999999},"last":10.390000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:20 2023","interfaces":[{"interface":"back","average":{"1min":12.436999999999999,"5min":11.664,"15min":11.535},"min":{"1min":6.9859999999999998,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":6.8680000000000003},{"interface":"front","average":{"1min":15.724,"5min":14.467000000000001,"15min":14.257},"min":{"1min":9.3049999999999997,"5min":6.609,"15min":6.609},"max":{"1min":30.751999999999999,"5min":30.751999999999999,"15min":30.751999999999999},"last":9.0220000000000002}]},{"osd":2,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":14.199,"5min":12.414,"15min":12.116},"min":{"1min":7.4530000000000003,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":24.254999999999999,"5min":24.254999999999999,"15min":24.254999999999999},"last":13.795999999999999},{"interface":"front","average":{"1min":15.834,"5min":15.787000000000001,"15min":15.779},"min":{"1min":9.625,"5min":9.625,"15min":9.625},"max":{"1min":25.428000000000001,"5min":25.428000000000001,"15min":25.428000000000001},"last":12.448}]},{"osd":3,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":10.042,"5min":11.02,"15min":11.183},"min":{"1min":6.415,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":32.927999999999997,"5min":32.927999999999997,"15min":32.927999999999997},"last":9.6449999999999996},{"interface":"front","average":{"1min":14.454000000000001,"5min":15.743,"15min":15.958},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":19.853000000000002,"5min":30.285,"15min":30.285},"last":16.638000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.544,"5min":19.591000000000001,"15min":19.599},"min":{"1min":10.619999999999999,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":40.25,"5min":64.474999999999994,"15min":64.474999999999994},"last":13.048999999999999},{"interface":"front","average":{"1min":15.550000000000001,"5min":16.079000000000001,"15min":16.167000000000002},"min":{"1min":6.9809999999999999,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":37.826999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":17.988}]},{"osd":6,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":20.481999999999999,"5min":19.119,"15min":18.891999999999999},"min":{"1min":14.273,"5min":14.273,"15min":14.273},"max":{"1min":35.241,"5min":35.241,"15min":35.241},"last":11.004},{"interface":"front","average":{"1min":14.154,"5min":14.051,"15min":14.034000000000001},"min":{"1min":8.8379999999999992,"5min":7.343,"15min":7.343},"max":{"1min":38.668999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":17.379999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":16.852,"5min":18.056000000000001,"15min":18.257000000000001},"min":{"1min":10.457000000000001,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":41.783999999999999,"5min":74.977999999999994,"15min":74.977999999999994},"last":14.423},{"interface":"front","average":{"1min":21.052,"5min":22.094000000000001,"15min":22.268000000000001},"min":{"1min":15.183999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":44.450000000000003,"5min":83.138000000000005,"15min":83.138000000000005},"last":11.701000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:26:54.713 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-10-30T06:26:54.713 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:26:54.714 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:26:54.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:26:54.753 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:54.752+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:54.799 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:24.778 96291== Warning: unimplemented fcntl command: 1036 2023-10-30T06:26:54.807 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.807 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.816 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:54.816+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:54.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:54.917 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:54.916+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:54.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:54.984 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:54.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:55.040 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:55.039+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:55.119 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:55.118+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:55.135 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:55.135 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:55.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:55.219 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:55.219+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:55.316 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:55.316 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:55.319 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:55.319+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:55.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:55.420 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:55.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:55.474 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:55.475 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:55.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:26:55.527 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:55.525+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:55.623 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:55.621+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:55.656 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:55.657 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:55.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:26:55.751 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:55.749+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:55.825 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:55.824+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:55.834 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:55.834 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:55.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:26:55.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:26:55.928 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:55.927+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:56.012 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.013 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:26:56.026 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:56.026+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:56.095 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:56.096 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 58, 2023-10-30T06:26:56.096 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:26:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:26:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:26:54.426158+0000", 2023-10-30T06:26:56.097 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:26:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:23:28.674155+0000", 2023-10-30T06:26:56.098 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:26:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2023-10-30T06:26:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:26:56.099 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2023-10-30T06:26:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2023-10-30T06:26:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:26:56.100 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:26:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:26:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:26:56.101 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:26:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:26:56.102 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:26:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:26:56.103 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:26:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:26:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:26:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:26:56.104 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:26:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:26:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:26:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:26:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.105 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:26:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:26:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:26:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:26:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:26:56.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:26:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:26:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:26:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:26:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:26:56.107 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:26:56.108 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:26:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:26:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:26:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:26:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:26:56.109 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:26:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:26:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:26:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:26:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:26:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:26:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:26:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:26:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:26:56.111 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:26:56.112 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:26:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:26:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:26:56.113 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:26:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:26:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:26:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:26:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:26:56.114 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:26:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:26:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:26:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:26:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:26:56.115 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.116 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:26:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:26:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:26:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:26:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:26:56.117 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:26:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:26:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:26:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:26:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:26:56.118 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:26:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:26:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:26:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:26:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.119 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:26:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:26:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:26:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:26:56.120 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:26:56.121 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:26:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:26:56.122 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:26:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:26:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:26:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:26:56.123 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:26:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:26:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:26:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:26:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:26:56.124 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:26:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:26:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:26:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:26:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:26:56.125 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:26:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:26:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:26:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:26:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:26:56.126 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:26:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:26:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:26:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:26:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:26:56.128 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:26:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:26:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:26:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:26:56.129 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:26:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:26:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:26:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:26:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:26:56.130 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:26:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:26:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:26:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:26:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:26:56.131 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:26:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:26:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:26:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:26:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:26:56.132 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:26:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:26:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:26:56.133 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:26:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:26:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:26:56.134 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:26:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:26:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:26:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:26:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:26:56.135 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:26:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:26:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:26:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:26:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:26:56.136 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.630000114440918, 2023-10-30T06:26:56.137 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.630000114440918, 2023-10-30T06:26:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:26:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2023-10-30T06:26:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2023-10-30T06:26:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:26:56.138 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:26:56.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:26:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:26:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.140 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:56.141 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:26:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.143 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.144 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:26:56.145 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:26:56.146 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:26:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.150 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:26:56.151 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:26:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:26:56.152 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:26:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:26:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:26:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.153 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:26:56.154 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.155 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:26:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 33, 2023-10-30T06:26:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:26:56.157 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:26:56.158 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.159 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:26:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.162 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:26:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.163 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:26:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.164 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:26:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:26:56.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:26:56.167 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.169 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.170 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 26, 2023-10-30T06:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.173 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.174 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.175 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.176 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:26:56.177 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:26:56.178 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.181 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:26:56.182 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.183 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.184 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.185 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:56.186 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.189 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.190 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.192 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.193 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.195 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.197 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:26:56.199 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:26:56.200 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.202 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:56.203 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.205 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.207 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:26:56.208 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.209 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.211 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.212 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.213 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.214 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.215 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:26:56.216 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.218 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:56.218 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:26:56.218 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.218 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.218 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.219 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.219 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:26:56.219 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.220 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.221 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.221 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.222 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.223 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:26:56.223 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.224 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:26:56.225 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.226 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.226 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.226 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.226 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.228 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.228 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:26:56.228 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:26:56.228 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.228 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:26:56.229 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.230 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.230 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.231 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:26:56.231 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:26:56.231 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.231 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:26:56.232 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:26:56.233 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.233 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.233 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.234 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.234 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:26:56.234 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.235 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.236 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:26:56.236 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.236 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.236 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:26:56.238 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.239 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:26:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:26:56.241 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.243 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:26:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.244 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:26:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:26:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.245 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:26:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:26:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:26:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:26:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.246 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.247 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:26:56.248 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:26:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:26:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:26:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:26:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:26:56.249 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:26:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:26:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.250 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:26:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.251 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.252 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:26:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:26:56.253 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:26:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:26:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:26:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.257 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:26:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:26:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:26:56.259 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:26:56.260 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:26:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:26:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:26:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:26:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:26:56.261 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:26:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:26:56.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:26:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.263 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.264 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:26:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.265 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:26:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:26:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.267 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:26:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:26:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:26:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.271 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:26:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:26:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.273 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:26:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.275 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:26:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.276 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:26:56.277 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:26:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:26:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:26:56.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:26:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:26:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:26:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:26:56.279 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.3", 2023-10-30T06:26:56.280 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:26:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2023-10-30T06:26:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 7 2023-10-30T06:26:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.281 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:26:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:26:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:26:56.282 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:26:56.283 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:26:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:26:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:26:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:26:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:26:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:26:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:26:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:26:56.285 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:26:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:26:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:26:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:26:56.286 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:26:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:26:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:26:56.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:56.287 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:56.289 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.289 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.290 INFO:tasks.thrashosds.thrasher:j is {'epoch': 58, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:26:54.426158+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:23:28.674155+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [1, 2]}], 'pg_upmap_items': [{'pgid': '2.3', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:26:56.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:26:56.291 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:56.288+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:56.292 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:26:56.390 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:56.389+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:56.421 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.422 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:26:56.491 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:56.489+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:56.527 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:26:56.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:26:56.557 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.557 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:26:56.607 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:56.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:56.642 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:26:56.657 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:26:56.657 INFO:tasks.mon_recovery:stopping all monitors 2023-10-30T06:26:56.657 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T06:26:56.658 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:26:56.675 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:26:56.663+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 96287) UID: 0 2023-10-30T06:26:56.676 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:26:56.665+0000 9f64640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:26:56.692 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:56.692+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:56.739 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.740 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:26:56.793 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:56.792+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:56.805 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:56.805 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:26:56.894 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:56.893+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:56.895 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:56.895 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:26:56.915 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:56.916 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:56.924 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:26:56.925 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:26:56.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:26:56.941 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2023-10-30T06:26:55.285161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221060,"kb_used_data":4888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753660,"statfs":{"total":773094113280,"available":772867747840,"internally_reserved":0,"allocated":5005312,"data_stored":2801060,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.751753"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.295296+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:26:50.295296+0000","last_peered":"2023-10-30T06:26:50.295296+0000","last_clean":"2023-10-30T06:26:50.295296+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:26:50.295296+0000","last_undegraded":"2023-10-30T06:26:50.295296+0000","last_fullsized":"2023-10-30T06:26:50.295296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.980342+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:26:48.980342+0000","last_peered":"2023-10-30T06:26:48.980342+0000","last_clean":"2023-10-30T06:26:48.980342+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:26:48.980342+0000","last_undegraded":"2023-10-30T06:26:48.980342+0000","last_fullsized":"2023-10-30T06:26:48.980342+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-30T06:26:52.293348+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:26:52.293348+0000","last_peered":"2023-10-30T06:26:52.293348+0000","last_clean":"2023-10-30T06:26:52.293348+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:26:52.293348+0000","last_undegraded":"2023-10-30T06:26:52.293348+0000","last_fullsized":"2023-10-30T06:26:52.293348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:48.899988+0000","last_change":"2023-10-30T06:24:19.260903+0000","last_active":"2023-10-30T06:26:48.899988+0000","last_peered":"2023-10-30T06:26:48.899988+0000","last_clean":"2023-10-30T06:26:48.899988+0000","last_became_active":"2023-10-30T06:24:19.234665+0000","last_became_peered":"2023-10-30T06:24:19.234665+0000","last_unstale":"2023-10-30T06:26:48.899988+0000","last_undegraded":"2023-10-30T06:26:48.899988+0000","last_fullsized":"2023-10-30T06:26:48.899988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:00:22.497040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.290952+0000","last_change":"2023-10-30T06:24:20.229924+0000","last_active":"2023-10-30T06:26:50.290952+0000","last_peered":"2023-10-30T06:26:50.290952+0000","last_clean":"2023-10-30T06:26:50.290952+0000","last_became_active":"2023-10-30T06:24:20.224560+0000","last_became_peered":"2023-10-30T06:24:20.224560+0000","last_unstale":"2023-10-30T06:26:50.290952+0000","last_undegraded":"2023-10-30T06:26:50.290952+0000","last_fullsized":"2023-10-30T06:26:50.290952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:19:56.643011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.393598+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:26:49.393598+0000","last_peered":"2023-10-30T06:26:49.393598+0000","last_clean":"2023-10-30T06:26:49.393598+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:26:49.393598+0000","last_undegraded":"2023-10-30T06:26:49.393598+0000","last_fullsized":"2023-10-30T06:26:49.393598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-10-30T06:26:50.286753+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:26:50.286753+0000","last_peered":"2023-10-30T06:26:50.286753+0000","last_clean":"2023-10-30T06:26:50.286753+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:26:50.286753+0000","last_undegraded":"2023-10-30T06:26:50.286753+0000","last_fullsized":"2023-10-30T06:26:50.286753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-30T06:26:49.395336+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:26:49.395336+0000","last_peered":"2023-10-30T06:26:49.395336+0000","last_clean":"2023-10-30T06:26:49.395336+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:26:49.395336+0000","last_undegraded":"2023-10-30T06:26:49.395336+0000","last_fullsized":"2023-10-30T06:26:49.395336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":150,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-10-30T06:26:52.295628+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:26:52.295628+0000","last_peered":"2023-10-30T06:26:52.295628+0000","last_clean":"2023-10-30T06:26:52.295628+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:26:52.295628+0000","last_undegraded":"2023-10-30T06:26:52.295628+0000","last_fullsized":"2023-10-30T06:26:52.295628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411364,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":656818,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":11.029999999999999,"5min":10.494,"15min":10.404},"min":{"1min":5.2720000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"last":15.856999999999999},{"interface":"front","average":{"1min":12.122999999999999,"5min":12.090999999999999,"15min":12.085000000000001},"min":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":35.398000000000003,"5min":46.798000000000002,"15min":46.798000000000002},"last":7.4059999999999997}]},{"osd":2,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":13.641,"15min":13.744999999999999},"min":{"1min":5.7350000000000003,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":44.537999999999997,"5min":52.552,"15min":52.552},"last":14.476000000000001},{"interface":"front","average":{"1min":17.143000000000001,"5min":15.545999999999999,"15min":15.279999999999999},"min":{"1min":7.391,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":55.585999999999999,"5min":55.585999999999999,"15min":55.585999999999999},"last":9.2059999999999995}]},{"osd":3,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":15.367000000000001,"5min":21.626000000000001,"15min":22.669},"min":{"1min":8.5980000000000008,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":32.259999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":15.214},{"interface":"front","average":{"1min":14.832000000000001,"5min":21.024000000000001,"15min":22.056000000000001},"min":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":38.987000000000002,"5min":124.075,"15min":124.075},"last":10.411}]},{"osd":4,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":13.747,"15min":13.699},"min":{"1min":7.6470000000000002,"5min":7.2359999999999998,"15min":7.2359999999999998},"max":{"1min":25.023,"5min":51.649000000000001,"15min":51.649000000000001},"last":8.5649999999999995},{"interface":"front","average":{"1min":18.245000000000001,"5min":18.047999999999998,"15min":18.015000000000001},"min":{"1min":10.759,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":44.973999999999997,"5min":44.973999999999997,"15min":44.973999999999997},"last":17.225999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":16.564,"5min":20.154,"15min":20.751999999999999},"min":{"1min":7.351,"5min":7.351,"15min":7.351},"max":{"1min":58.648000000000003,"5min":123.754,"15min":123.754},"last":11.435},{"interface":"front","average":{"1min":14.667999999999999,"5min":21.396999999999998,"15min":22.518999999999998},"min":{"1min":7.6619999999999999,"5min":7.6619999999999999,"15min":7.6619999999999999},"max":{"1min":41.558999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":13.851000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":17.972000000000001,"5min":20.760000000000002,"15min":21.224},"min":{"1min":9.359,"5min":9.359,"15min":9.359},"max":{"1min":39.860999999999997,"5min":66.400999999999996,"15min":66.400999999999996},"last":12.01},{"interface":"front","average":{"1min":16.838999999999999,"5min":19.254999999999999,"15min":19.657},"min":{"1min":9.6969999999999992,"5min":8.2070000000000007,"15min":8.2070000000000007},"max":{"1min":37.027999999999999,"5min":53.149000000000001,"15min":53.149000000000001},"last":12.611000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:18 2023","interfaces":[{"interface":"back","average":{"1min":18.468,"5min":20.904,"15min":21.309999999999999},"min":{"1min":10.083,"5min":10.083,"15min":10.083},"max":{"1min":51.375,"5min":51.375,"15min":51.375},"last":18.904},{"interface":"front","average":{"1min":19.795999999999999,"5min":21.234999999999999,"15min":21.475000000000001},"min":{"1min":10.004,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":54.439,"5min":69.991,"15min":69.991},"last":16.504000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476773,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.561,"5min":13.877000000000001,"15min":14.429},"min":{"1min":4.9980000000000002,"5min":4.8300000000000001,"15min":4.8300000000000001},"max":{"1min":37.701999999999998,"5min":94.343000000000004,"15min":94.343000000000004},"last":14.741},{"interface":"front","average":{"1min":11.94,"5min":14.991,"15min":15.499000000000001},"min":{"1min":5.8700000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":49.045999999999999,"5min":122.054,"15min":122.054},"last":5.6390000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.071,"5min":10.667,"15min":10.766},"min":{"1min":5.8789999999999996,"5min":5.8789999999999996,"15min":5.8789999999999996},"max":{"1min":16.489999999999998,"5min":18.992999999999999,"15min":18.992999999999999},"last":9.9149999999999991},{"interface":"front","average":{"1min":13.263,"5min":29.742999999999999,"15min":32.490000000000002},"min":{"1min":8.984,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":19.466999999999999,"5min":387.15600000000001,"15min":387.15600000000001},"last":13.932}]},{"osd":2,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.083,"5min":18.381,"15min":19.431000000000001},"min":{"1min":6.5800000000000001,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":31.495999999999999,"5min":156.90299999999999,"15min":156.90299999999999},"last":6.8129999999999997},{"interface":"front","average":{"1min":14.241,"5min":19.315000000000001,"15min":20.161000000000001},"min":{"1min":7.7249999999999996,"5min":7.7249999999999996,"15min":7.7249999999999996},"max":{"1min":20.09,"5min":143.578,"15min":143.578},"last":15.561}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.124000000000001,"5min":20.594999999999999,"15min":21.507000000000001},"min":{"1min":9.6370000000000005,"5min":9.6370000000000005,"15min":9.6370000000000005},"max":{"1min":21.492999999999999,"5min":141.18799999999999,"15min":141.18799999999999},"last":16.382000000000001},{"interface":"front","average":{"1min":18.352,"5min":24.007000000000001,"15min":24.949000000000002},"min":{"1min":13.48,"5min":13.48,"15min":13.48},"max":{"1min":30.067,"5min":161.32900000000001,"15min":161.32900000000001},"last":17.221}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":11.082000000000001,"5min":14.252000000000001,"15min":14.779999999999999},"min":{"1min":6.2240000000000002,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":18.454000000000001,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.1020000000000003},{"interface":"front","average":{"1min":13.582000000000001,"5min":15.278,"15min":15.56},"min":{"1min":8.3230000000000004,"5min":8.3230000000000004,"15min":8.3230000000000004},"max":{"1min":20.756,"5min":58.613999999999997,"15min":58.613999999999997},"last":12.173}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":13.81,"5min":17.134,"15min":17.687999999999999},"min":{"1min":7.383,"5min":7.383,"15min":7.383},"max":{"1min":20.297000000000001,"5min":71.757000000000005,"15min":71.757000000000005},"last":8.9179999999999993},{"interface":"front","average":{"1min":17.872,"5min":20.984000000000002,"15min":21.501999999999999},"min":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"max":{"1min":24.132000000000001,"5min":80.119,"15min":80.119},"last":17.951000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.315000000000001,"5min":19.055,"15min":19.512},"min":{"1min":8.9610000000000003,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":29.869,"5min":81.247,"15min":81.247},"last":12.864000000000001},{"interface":"front","average":{"1min":16.158000000000001,"5min":18.501999999999999,"15min":18.891999999999999},"min":{"1min":10.763,"5min":9.0600000000000005,"15min":9.0600000000000005},"max":{"1min":27.731999999999999,"5min":70.989000000000004,"15min":70.989000000000004},"last":11.109}]}]},{"osd":3,"up_from":16,"seq":68719476773,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":10.17,"5min":10.882,"15min":11},"min":{"1min":6.6020000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":26.021000000000001,"5min":26.021000000000001,"15min":26.021000000000001},"last":12.115},{"interface":"front","average":{"1min":11.24,"5min":10.454000000000001,"15min":10.323},"min":{"1min":4.7220000000000004,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":27.683,"5min":27.683,"15min":27.683},"last":8.1539999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":12.688000000000001,"5min":23.998999999999999,"15min":25.884},"min":{"1min":7.4039999999999999,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":22.036000000000001,"5min":165.76400000000001,"15min":165.76400000000001},"last":18.942},{"interface":"front","average":{"1min":12.916,"5min":25.045999999999999,"15min":27.068000000000001},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24.646999999999998,"5min":190.34200000000001,"15min":190.34200000000001},"last":18.216000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.867000000000001,"5min":17.914999999999999,"15min":18.257000000000001},"min":{"1min":10.250999999999999,"5min":10.250999999999999,"15min":10.250999999999999},"max":{"1min":30.335000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":11.191000000000001},{"interface":"front","average":{"1min":14.234,"5min":16.606000000000002,"15min":17.001000000000001},"min":{"1min":9.1639999999999997,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":23.334,"5min":33.060000000000002,"15min":33.060000000000002},"last":14.647}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":15.425000000000001,"5min":19.581,"15min":20.273},"min":{"1min":10.442,"5min":9.9420000000000002,"15min":9.9420000000000002},"max":{"1min":23.326000000000001,"5min":42.274999999999999,"15min":42.274999999999999},"last":15.43},{"interface":"front","average":{"1min":9.7880000000000003,"5min":11.247999999999999,"15min":11.492000000000001},"min":{"1min":5.7969999999999997,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":19.716000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":13.555}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.015000000000001,"5min":18.329999999999998,"15min":19.216000000000001},"min":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"max":{"1min":22.614000000000001,"5min":56.796999999999997,"15min":56.796999999999997},"last":9.4770000000000003},{"interface":"front","average":{"1min":15.929,"5min":19.824999999999999,"15min":20.474},"min":{"1min":8.6620000000000008,"5min":8.6620000000000008,"15min":8.6620000000000008},"max":{"1min":28.501000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":17.27}]},{"osd":6,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":19.527999999999999,"5min":27.216000000000001,"15min":28.497},"min":{"1min":10.875,"5min":10.875,"15min":10.875},"max":{"1min":41.771000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":16.042999999999999},{"interface":"front","average":{"1min":14.746,"5min":15.343999999999999,"15min":15.444000000000001},"min":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"max":{"1min":40.128999999999998,"5min":40.128999999999998,"15min":40.128999999999998},"last":6.5510000000000002}]},{"osd":7,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":16.882999999999999,"5min":17.765000000000001,"15min":17.911999999999999},"min":{"1min":10.404,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":31.125,"5min":38.887999999999998,"15min":38.887999999999998},"last":44.746000000000002},{"interface":"front","average":{"1min":19.367999999999999,"5min":27.565000000000001,"15min":28.931000000000001},"min":{"1min":12.272,"5min":10.025,"15min":10.025},"max":{"1min":29.981999999999999,"5min":170.232,"15min":170.232},"last":42.689999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.563000000000001,"5min":24.759,"15min":26.792000000000002},"min":{"1min":5.0410000000000004,"5min":5.0410000000000004,"15min":5.0410000000000004},"max":{"1min":51.853000000000002,"5min":258.66199999999998,"15min":258.66199999999998},"last":9.8059999999999992},{"interface":"front","average":{"1min":12.401,"5min":21.513999999999999,"15min":23.033000000000001},"min":{"1min":6.6879999999999997,"5min":6.016,"15min":6.016},"max":{"1min":49.905000000000001,"5min":203.38499999999999,"15min":203.38499999999999},"last":12.289999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":12.252000000000001,"5min":15.499000000000001,"15min":16.039999999999999},"min":{"1min":6.6790000000000003,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":20.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":22.904},{"interface":"front","average":{"1min":11.571999999999999,"5min":14.185,"15min":14.621},"min":{"1min":8.8179999999999996,"5min":6.8360000000000003,"15min":6.8360000000000003},"max":{"1min":16.605,"5min":37.390999999999998,"15min":37.390999999999998},"last":15.984999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":10.577999999999999,"5min":17.576000000000001,"15min":18.742000000000001},"min":{"1min":6.9020000000000001,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":14.788,"5min":183.38900000000001,"15min":183.38900000000001},"last":8.0210000000000008},{"interface":"front","average":{"1min":14.112,"5min":21.276,"15min":22.469999999999999},"min":{"1min":8.2420000000000009,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.007000000000001,"5min":189.374,"15min":189.374},"last":24.317}]},{"osd":4,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.077999999999999,"5min":20.582000000000001,"15min":21.5},"min":{"1min":11.079000000000001,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":27.879999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":13.766999999999999},{"interface":"front","average":{"1min":12.926,"5min":18.677,"15min":19.635000000000002},"min":{"1min":7.3929999999999998,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":26.975999999999999,"5min":167.67599999999999,"15min":167.67599999999999},"last":11.717000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":16.343,"5min":25.315999999999999,"15min":26.812000000000001},"min":{"1min":10.385,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":21.745000000000001,"5min":215.80500000000001,"15min":215.80500000000001},"last":19.259},{"interface":"front","average":{"1min":15.525,"5min":24.969000000000001,"15min":26.542999999999999},"min":{"1min":10.387,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":19.696999999999999,"5min":187.863,"15min":187.863},"last":18.067}]},{"osd":6,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":15.364000000000001,"5min":24.882000000000001,"15min":26.468},"min":{"1min":11.069000000000001,"5min":10.587999999999999,"15min":10.587999999999999},"max":{"1min":20.021999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":14.771000000000001},{"interface":"front","average":{"1min":18.702000000000002,"5min":28.018999999999998,"15min":29.571999999999999},"min":{"1min":11.772,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":25.587,"5min":275.92200000000003,"15min":275.92200000000003},"last":28.056999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:14 2023","interfaces":[{"interface":"back","average":{"1min":17.663,"5min":25.314,"15min":26.588999999999999},"min":{"1min":13.518000000000001,"5min":13.518000000000001,"15min":13.518000000000001},"max":{"1min":24.042999999999999,"5min":170.43600000000001,"15min":170.43600000000001},"last":25.423999999999999},{"interface":"front","average":{"1min":17.009,"5min":22.231999999999999,"15min":23.102},"min":{"1min":10.459,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":24.635000000000002,"5min":157.00200000000001,"15min":157.00200000000001},"last":20.588999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476771,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":253441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":12.038,"5min":10.962999999999999,"15min":10.784000000000001},"min":{"1min":4.5149999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":39.700000000000003,"5min":39.700000000000003,"15min":39.700000000000003},"last":9.1899999999999995},{"interface":"front","average":{"1min":11.220000000000001,"5min":11.77,"15min":11.862},"min":{"1min":5.2050000000000001,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":28.358000000000001,"5min":30.788,"15min":30.788},"last":8.2899999999999991}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.443,"5min":18.129999999999999,"15min":19.411000000000001},"min":{"1min":6.3289999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":20.760999999999999,"5min":127.95399999999999,"15min":127.95399999999999},"last":7.1769999999999996},{"interface":"front","average":{"1min":12.442,"5min":14.385999999999999,"15min":14.710000000000001},"min":{"1min":6.8070000000000004,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":23.695,"5min":35.834000000000003,"15min":35.834000000000003},"last":11.340999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.311,"5min":20.981999999999999,"15min":22.094000000000001},"min":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":34.953000000000003,"5min":99.790999999999997,"15min":99.790999999999997},"last":10.712},{"interface":"front","average":{"1min":14.433,"5min":16.257999999999999,"15min":16.562000000000001},"min":{"1min":9.5980000000000008,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":33.680999999999997,"5min":33.680999999999997,"15min":33.680999999999997},"last":18.760000000000002}]},{"osd":4,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.423999999999999,"5min":15.603999999999999,"15min":16.300999999999998},"min":{"1min":6.8150000000000004,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":19.856999999999999,"5min":41.134999999999998,"15min":41.134999999999998},"last":10.109},{"interface":"front","average":{"1min":15.153,"5min":23.329000000000001,"15min":24.692},"min":{"1min":8.5579999999999998,"5min":8.5579999999999998,"15min":8.5579999999999998},"max":{"1min":23.863,"5min":103.705,"15min":103.705},"last":17.050000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.125,"5min":18.056000000000001,"15min":18.710999999999999},"min":{"1min":8.4629999999999992,"5min":8.4629999999999992,"15min":8.4629999999999992},"max":{"1min":26.533999999999999,"5min":43.847000000000001,"15min":43.847000000000001},"last":13.653},{"interface":"front","average":{"1min":16.975999999999999,"5min":26.073,"15min":27.588999999999999},"min":{"1min":9.6270000000000007,"5min":9.6270000000000007,"15min":9.6270000000000007},"max":{"1min":28.102,"5min":98.707999999999998,"15min":98.707999999999998},"last":12.885}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":18.401,"5min":20.082999999999998,"15min":20.363},"min":{"1min":8.7550000000000008,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":55.631,"5min":55.631,"15min":55.631},"last":15.637},{"interface":"front","average":{"1min":17.962,"5min":21.433,"15min":22.010999999999999},"min":{"1min":10.673999999999999,"5min":10.673999999999999,"15min":10.673999999999999},"max":{"1min":54.656999999999996,"5min":54.656999999999996,"15min":54.656999999999996},"last":15.012}]},{"osd":7,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":17.059000000000001,"5min":21.111000000000001,"15min":21.786000000000001},"min":{"1min":12.289,"5min":11.917999999999999,"15min":11.917999999999999},"max":{"1min":23.265000000000001,"5min":45.927999999999997,"15min":45.927999999999997},"last":19.673999999999999},{"interface":"front","average":{"1min":17.498999999999999,"5min":22.126000000000001,"15min":22.896999999999998},"min":{"1min":11.305999999999999,"5min":11.305999999999999,"15min":11.305999999999999},"max":{"1min":28.651,"5min":45.130000000000003,"15min":45.130000000000003},"last":16.356999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":9.0389999999999997,"5min":14.882999999999999,"15min":15.856999999999999},"min":{"1min":4.431,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":17.584,"5min":131.477,"15min":131.477},"last":6.5090000000000003},{"interface":"front","average":{"1min":9.7940000000000005,"5min":15.146000000000001,"15min":16.038},"min":{"1min":6.0839999999999996,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":20.175999999999998,"5min":129.386,"15min":129.386},"last":7.7370000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":10.300000000000001,"5min":10.855,"15min":10.946999999999999},"min":{"1min":6.5270000000000001,"5min":6.5270000000000001,"15min":6.5270000000000001},"max":{"1min":14.603,"5min":25.774000000000001,"15min":25.774000000000001},"last":13.699},{"interface":"front","average":{"1min":16.196000000000002,"5min":18.829999999999998,"15min":19.268999999999998},"min":{"1min":11.680999999999999,"5min":9.9960000000000004,"15min":9.9960000000000004},"max":{"1min":27.390999999999998,"5min":91.295000000000002,"15min":91.295000000000002},"last":15.038}]},{"osd":2,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.880000000000001,"5min":17.151,"15min":17.863},"min":{"1min":6.0830000000000002,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":46.811,"5min":134.21199999999999,"15min":134.21199999999999},"last":22.661999999999999},{"interface":"front","average":{"1min":14.634,"5min":14.634,"15min":14.634},"min":{"1min":5.992,"5min":5.992,"15min":5.992},"max":{"1min":50.450000000000003,"5min":50.450000000000003,"15min":50.450000000000003},"last":18.574000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":12.888,"5min":19.111999999999998,"15min":20.149999999999999},"min":{"1min":6.758,"5min":6.758,"15min":6.758},"max":{"1min":22.356999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":17.547999999999998},{"interface":"front","average":{"1min":15.807,"5min":20.111000000000001,"15min":20.829000000000001},"min":{"1min":9.1150000000000002,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":25.867000000000001,"5min":124.684,"15min":124.684},"last":14.35}]},{"osd":4,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":12.601000000000001,"5min":13.630000000000001,"15min":13.802},"min":{"1min":8.5579999999999998,"5min":6.9610000000000003,"15min":6.9610000000000003},"max":{"1min":17.361999999999998,"5min":24.963000000000001,"15min":24.963000000000001},"last":20.908000000000001},{"interface":"front","average":{"1min":16.216000000000001,"5min":15.724,"15min":15.641999999999999},"min":{"1min":11.066000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":26.544,"5min":26.544,"15min":26.544},"last":25.838999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:26:13 2023","interfaces":[{"interface":"back","average":{"1min":13.038,"5min":19.187000000000001,"15min":20.212},"min":{"1min":7.016,"5min":7.016,"15min":7.016},"max":{"1min":18.068999999999999,"5min":125.792,"15min":125.792},"last":26.137},{"interface":"front","average":{"1min":16.5,"5min":16.905000000000001,"15min":16.972999999999999},"min":{"1min":9.8420000000000005,"5min":9.8420000000000005,"15min":9.8420000000000005},"max":{"1min":24.768000000000001,"5min":28.591999999999999,"15min":28.591999999999999},"last":21.574999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:26:16 2023","interfaces":[{"interface":"back","average":{"1min":16.189,"5min":28.972000000000001,"15min":31.102},"min":{"1min":7.6020000000000003,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":24.219000000000001,"5min":327.29599999999999,"15min":327.29599999999999},"last":15.619},{"interface":"front","average":{"1min":14.366,"5min":30.007999999999999,"15min":32.615000000000002},"min":{"1min":7.5499999999999998,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":18.904,"5min":323.82400000000001,"15min":323.82400000000001},"last":20.329999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":645744,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":10.606999999999999,"5min":14.191000000000001,"15min":14.788},"min":{"1min":5.5609999999999999,"5min":5.4829999999999997,"15min":5.4829999999999997},"max":{"1min":28.821999999999999,"5min":68.513999999999996,"15min":68.513999999999996},"last":11.003},{"interface":"front","average":{"1min":10.786,"5min":15.789999999999999,"15min":16.623999999999999},"min":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":27.027999999999999,"5min":88.816000000000003,"15min":88.816000000000003},"last":6.4530000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":11.936999999999999,"5min":16.495000000000001,"15min":17.254999999999999},"min":{"1min":7.3710000000000004,"5min":6.9269999999999996,"15min":6.9269999999999996},"max":{"1min":19.015999999999998,"5min":87.515000000000001,"15min":87.515000000000001},"last":30.579999999999998},{"interface":"front","average":{"1min":12.638999999999999,"5min":16.937999999999999,"15min":17.654},"min":{"1min":6.5549999999999997,"5min":6.21,"15min":6.21},"max":{"1min":26.303000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":28.934999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":15.009,"5min":17.692,"15min":18.138999999999999},"min":{"1min":10.311,"5min":6.6550000000000002,"15min":6.6550000000000002},"max":{"1min":23.599,"5min":113.943,"15min":113.943},"last":25.640000000000001},{"interface":"front","average":{"1min":11.092000000000001,"5min":14.593999999999999,"15min":15.178000000000001},"min":{"1min":5.7949999999999999,"5min":5.7949999999999999,"15min":5.7949999999999999},"max":{"1min":22.634,"5min":80.004000000000005,"15min":80.004000000000005},"last":33.826000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":16.504999999999999,"5min":20.773,"15min":21.484999999999999},"min":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":87.866,"5min":87.866,"15min":87.866},"last":8.0009999999999994},{"interface":"front","average":{"1min":19.504999999999999,"5min":23.062000000000001,"15min":23.655000000000001},"min":{"1min":11.359999999999999,"5min":9.6720000000000006,"15min":9.6720000000000006},"max":{"1min":89.334000000000003,"5min":91.183000000000007,"15min":91.183000000000007},"last":11.882}]},{"osd":4,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":12.736000000000001,"5min":12.737,"15min":12.737},"min":{"1min":7.9050000000000002,"5min":7.0099999999999998,"15min":7.0099999999999998},"max":{"1min":20.556999999999999,"5min":25.195,"15min":25.195},"last":14.602},{"interface":"front","average":{"1min":14.071,"5min":16.562999999999999,"15min":16.978000000000002},"min":{"1min":7.2279999999999998,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":19.228000000000002,"5min":59.609000000000002,"15min":59.609000000000002},"last":23.651}]},{"osd":6,"last update":"Mon Oct 30 06:26:17 2023","interfaces":[{"interface":"back","average":{"1min":14.146000000000001,"5min":15.726000000000001,"15min":15.99},"min":{"1min":7.6980000000000004,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":46.561999999999998,"5min":66.765000000000001,"15min":66.765000000000001},"last":9.2989999999999995},{"interface":"front","average":{"1min":17.895,"5min":20.585000000000001,"15min":21.033000000000001},"min":{"1min":11.125999999999999,"5min":11.125999999999999,"15min":11.125999999999999},"max":{"1min":39.637999999999998,"5min":71.771000000000001,"15min":71.771000000000001},"last":11.585000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:26:12 2023","interfaces":[{"interface":"back","average":{"1min":17.015000000000001,"5min":18.721,"15min":19.004999999999999},"min":{"1min":10.923,"5min":9.8770000000000007,"15min":9.8770000000000007},"max":{"1min":25.045999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":15.161},{"interface":"front","average":{"1min":15.677,"5min":17.84,"15min":18.199999999999999},"min":{"1min":8.1170000000000009,"5min":8.1170000000000009,"15min":8.1170000000000009},"max":{"1min":21.984000000000002,"5min":42.531999999999996,"15min":42.531999999999996},"last":29.597999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509476,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":247904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":8.6489999999999991,"5min":9.1769999999999996,"15min":9.2650000000000006},"min":{"1min":4.6660000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":26.984999999999999,"5min":26.984999999999999,"15min":26.984999999999999},"last":8.0329999999999995},{"interface":"front","average":{"1min":11.923999999999999,"5min":10.831,"15min":10.648999999999999},"min":{"1min":6.4500000000000002,"5min":6.2850000000000001,"15min":6.2850000000000001},"max":{"1min":25.934999999999999,"5min":25.934999999999999,"15min":25.934999999999999},"last":10.390000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:26:20 2023","interfaces":[{"interface":"back","average":{"1min":12.436999999999999,"5min":11.664,"15min":11.535},"min":{"1min":6.9859999999999998,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":24.265000000000001,"5min":24.265000000000001,"15min":24.265000000000001},"last":6.8680000000000003},{"interface":"front","average":{"1min":15.724,"5min":14.467000000000001,"15min":14.257},"min":{"1min":9.3049999999999997,"5min":6.609,"15min":6.609},"max":{"1min":30.751999999999999,"5min":30.751999999999999,"15min":30.751999999999999},"last":9.0220000000000002}]},{"osd":2,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":14.199,"5min":12.414,"15min":12.116},"min":{"1min":7.4530000000000003,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":24.254999999999999,"5min":24.254999999999999,"15min":24.254999999999999},"last":13.795999999999999},{"interface":"front","average":{"1min":15.834,"5min":15.787000000000001,"15min":15.779},"min":{"1min":9.625,"5min":9.625,"15min":9.625},"max":{"1min":25.428000000000001,"5min":25.428000000000001,"15min":25.428000000000001},"last":12.448}]},{"osd":3,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":10.042,"5min":11.02,"15min":11.183},"min":{"1min":6.415,"5min":5.6689999999999996,"15min":5.6689999999999996},"max":{"1min":32.927999999999997,"5min":32.927999999999997,"15min":32.927999999999997},"last":9.6449999999999996},{"interface":"front","average":{"1min":14.454000000000001,"5min":15.743,"15min":15.958},"min":{"1min":9.5359999999999996,"5min":9.5359999999999996,"15min":9.5359999999999996},"max":{"1min":19.853000000000002,"5min":30.285,"15min":30.285},"last":16.638000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":19.544,"5min":19.591000000000001,"15min":19.599},"min":{"1min":10.619999999999999,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":40.25,"5min":64.474999999999994,"15min":64.474999999999994},"last":13.048999999999999},{"interface":"front","average":{"1min":15.550000000000001,"5min":16.079000000000001,"15min":16.167000000000002},"min":{"1min":6.9809999999999999,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":37.826999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":17.988}]},{"osd":6,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":20.481999999999999,"5min":19.119,"15min":18.891999999999999},"min":{"1min":14.273,"5min":14.273,"15min":14.273},"max":{"1min":35.241,"5min":35.241,"15min":35.241},"last":11.004},{"interface":"front","average":{"1min":14.154,"5min":14.051,"15min":14.034000000000001},"min":{"1min":8.8379999999999992,"5min":7.343,"15min":7.343},"max":{"1min":38.668999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":17.379999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:26:15 2023","interfaces":[{"interface":"back","average":{"1min":16.852,"5min":18.056000000000001,"15min":18.257000000000001},"min":{"1min":10.457000000000001,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":41.783999999999999,"5min":74.977999999999994,"15min":74.977999999999994},"last":14.423},{"interface":"front","average":{"1min":21.052,"5min":22.094000000000001,"15min":22.268000000000001},"min":{"1min":15.183999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":44.450000000000003,"5min":83.138000000000005,"15min":83.138000000000005},"last":11.701000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:26:56.943 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [7, 4, 6, 1] 2023-10-30T06:26:56.944 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '7', '4', '6', '1'] 2023-10-30T06:26:56.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.2 7 4 6 1 2023-10-30T06:26:56.982 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:56.983 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.012 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:57.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:57.071 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:57.071 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.096 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:57.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:57.114 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:26:57.114 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:26:57.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:26:57.183 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:26:57.183 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.184 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:27.187 96291== 2023-10-30T06:26:57.184 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:27.187 96291== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:26:57.193 INFO:tasks.ceph.mon.b.smithi112.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:26:57.197 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:57.197+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:57.272 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:26:57.273 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.273 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:57.274 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:57.274 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:57.274 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:57.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:26:57.325 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:57.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:57.354 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:26:57.355 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.399 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:57.397+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:57.436 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:26:57.436 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:26:57.471 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:57.472 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:57.472 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:57.472 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:57.472 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:57.480 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:26:57.500 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:26:57.498+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:26:57.600 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:57.599+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:57.654 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:57.655 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:57.655 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:57.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:26:57.702 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:57.700+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:57.793 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:57.793 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:57.793 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:57.793 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:57.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:26:57.803 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:57.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:57.903 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:57.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:57.973 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:57.973 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:57.974 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:57.974 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:57.974 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:57.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:26:58.023 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:58.021+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:58.105 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:26:58.104+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:26:58.201 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:58.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:26:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 164.86523460800001, 2023-10-30T06:26:58.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:26:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:26:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.205 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:58.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:26:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:26:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:26:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:26:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:26:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:26:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:26:58.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:26:58.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:26:58.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:26:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:26:58.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:26:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:26:58.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:26:58.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:26:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:26:58.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:26:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:26:58.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:26:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:26:58.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:26:58.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:26:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:26:58.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.54407941700001, 2023-10-30T06:26:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:26:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:26:58.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:58.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:26:58.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:26:58.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:26:58.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:26:58.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:26:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:26:58.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:26:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:26:58.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:26:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:26:58.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:26:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:26:58.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:26:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:26:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:26:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:26:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:26:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:26:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:26:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:26:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:26:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.53983486000001, 2023-10-30T06:26:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:26:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:26:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:26:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:26:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:26:58.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:26:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:26:58.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:26:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:26:58.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:26:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:26:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:26:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:26:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:26:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:26:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:26:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:26:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:26:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:26:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:26:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:26:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.454376435, 2023-10-30T06:26:58.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:26:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.257 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:26:58.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:26:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:26:58.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:26:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:26:58.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:26:58.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:26:58.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:26:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:26:58.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:26:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:26:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:26:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:26:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:26:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:26:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:26:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:26:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:26:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:26:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:26:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.271 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.272 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.18513321899999, 2023-10-30T06:26:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:26:58.273 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.274 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:26:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:26:58.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:26:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:26:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:26:58.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:26:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:26:58.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:26:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:26:58.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:26:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:26:58.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:26:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:26:58.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:26:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:26:58.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:26:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:26:58.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:26:58.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:26:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:26:58.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.18430180799999, 2023-10-30T06:26:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:26:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.290 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:26:58.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:26:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:26:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:26:58.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:26:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:26:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:26:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:26:58.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:26:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:26:58.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:26:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:26:58.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:26:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:26:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:26:58.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:26:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:26:58.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:26:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:26:58.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:26:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.183634681, 2023-10-30T06:26:58.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:26:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:26:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:58.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:26:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:26:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:26:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:26:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:26:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:26:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:26:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:26:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:26:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:26:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:26:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:26:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:26:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:26:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:26:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:26:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:26:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:26:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:26:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:26:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:26:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.16136061, 2023-10-30T06:26:58.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:26:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.324 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:26:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:26:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:26:58.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:26:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:26:58.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:26:58.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:26:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:26:58.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:26:58.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:26:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:26:58.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:26:58.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:26:58.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:26:58.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:26:58.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:26:58.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:26:58.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:26:58.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:58.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.160712754, 2023-10-30T06:26:58.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:26:58.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:26:58.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.342 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:58.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:26:58.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:26:58.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:26:58.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:26:58.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:26:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:26:58.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:26:58.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:26:58.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:26:58.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:26:58.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:26:58.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:26:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:26:58.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:26:58.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:26:58.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:26:58.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:26:58.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:58.357 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 162.16002668100001, 2023-10-30T06:26:58.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:26:58.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:26:58.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:26:58.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:26:58.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:26:58.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:26:58.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:26:58.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:26:58.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:26:58.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:26:58.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:26:58.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:26:58.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:26:58.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:26:58.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:26:58.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:26:58.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:26:58.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:26:58.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:26:58.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:26:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:26:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:26:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:26:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:58.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:58.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 161.089144713, 2023-10-30T06:26:58.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:26:58.374 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.374 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.375 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:26:58.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:58.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:26:58.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:26:58.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:26:58.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:26:58.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:26:58.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:26:58.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:26:58.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:26:58.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:26:58.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:26:58.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:26:58.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:26:58.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:26:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:26:58.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:26:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:26:58.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:26:58.389 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.390 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 161.08848074700001, 2023-10-30T06:26:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:26:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.391 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.392 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:26:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:26:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:26:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:26:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:26:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:26:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:26:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:26:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:26:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:26:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:26:58.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:26:58.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:26:58.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:26:58.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:26:58.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:26:58.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.405 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.405 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.406 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:26:58.406 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:58.406 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 161.06002693799999, 2023-10-30T06:26:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:26:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.407 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.408 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:26:58.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.408 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:58.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:26:58.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:26:58.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:26:58.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:26:58.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:26:58.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:26:58.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:26:58.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:26:58.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:26:58.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:26:58.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:26:58.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:26:58.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:26:58.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:26:58.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:26:58.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:26:58.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:26:58.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:26:58.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:26:58.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:58.424 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.718260445, 2023-10-30T06:26:58.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:26:58.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:58.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:26:58.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:26:58.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:26:58.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:26:58.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:26:58.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:26:58.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:26:58.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:26:58.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:26:58.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:26:58.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:26:58.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:26:58.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:26:58.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:26:58.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:26:58.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.439 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:58.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.71751177499999, 2023-10-30T06:26:58.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:26:58.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:26:58.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.441 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:58.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:26:58.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:26:58.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:26:58.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:26:58.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:26:58.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:26:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:26:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:26:58.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:26:58.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:26:58.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:26:58.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:26:58.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:26:58.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:26:58.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:26:58.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:26:58.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:26:58.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:26:58.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:26:58.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:26:58.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:26:58.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:58.456 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.71684339199999, 2023-10-30T06:26:58.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.457 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:26:58.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:58.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:26:58.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:26:58.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:26:58.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:26:58.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:26:58.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:26:58.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:26:58.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:26:58.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:26:58.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:26:58.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:26:58.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:26:58.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:26:58.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:26:58.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:26:58.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:26:58.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:26:58.470 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.470 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.471 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.471 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:58.472 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.70699271800001, 2023-10-30T06:26:58.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:26:58.472 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.472 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.472 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.473 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.473 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:26:58.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.473 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:58.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:26:58.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:26:58.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:26:58.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:26:58.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:26:58.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:26:58.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:26:58.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:26:58.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:26:58.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:26:58.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:26:58.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:26:58.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:26:58.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:26:58.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:26:58.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:26:58.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:26:58.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:26:58.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:26:58.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:26:58.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:26:58.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.489 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.489 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:58.489 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.70631890300001, 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:26:58.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:26:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:26:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:26:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:26:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:26:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:26:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:26:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:26:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:26:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:26:58.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:26:58.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:26:58.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:26:58.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:26:58.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:26:58.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:26:58.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:26:58.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:26:58.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.504 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.505 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:58.505 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.70567052000001, 2023-10-30T06:26:58.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:26:58.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.505 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.506 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.506 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:58.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:26:58.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:26:58.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:26:58.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:26:58.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:26:58.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:26:58.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:26:58.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:26:58.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:26:58.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:26:58.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:26:58.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:26:58.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:26:58.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:26:58.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:26:58.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:26:58.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:26:58.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:26:58.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:26:58.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:26:58.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:26:58.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:26:58.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:26:58.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.522 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:26:58.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:58.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 153.70155405400001, 2023-10-30T06:26:58.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:26:58.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:58.523 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:26:58.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:58.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:26:58.524 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:26:58.524 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:26:58.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.525 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:58.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:58.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:58.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:58.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:58.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:26:58.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:26:58.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:58.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:26:58.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:26:58.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:58.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:26:58.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:26:58.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:26:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:26:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:26:58.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:26:58.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:26:58.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:26:58.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:26:58.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:26:58.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:58.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:58.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:58.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:58.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:58.538 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:58.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:26:58.540 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:58.538+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:58.641 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:58.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:58.706 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:58.707 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:58.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:26:58.741 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:58.740+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:58.842 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:58.841+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:58.844 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:58.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:58.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:26:58.944 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:58.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:59.022 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:59.022 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:59.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:59.023 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:26:59.023 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:59.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:26:59.073 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:59.071+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:59.145 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:59.144+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:26:59.201 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:59.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:59.202 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:26:59.202 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:59.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:26:59.254 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:26:59.251+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:26:59.347 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:59.346+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:59.378 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:26:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:26:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:26:59.379 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:59.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:26:59.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:26:59.449 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:59.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:59.518 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:26:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:26:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:26:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:26:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:26:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 83.226740429000003, 2023-10-30T06:26:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:26:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:26:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:26:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:26:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:26:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:26:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:26:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:26:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:26:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:26:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:26:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:26:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:26:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:26:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:26:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:26:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:26:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:26:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:26:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:26:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:26:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:26:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:26:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:26:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:26:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:26:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:26:59.530 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:26:59.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:26:59.577 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:26:59.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:26:59.649 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-30T06:26:59.672 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:59.673 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:59.673 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:26:59.673 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:59.681 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:26:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:26:59.682 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:26:59.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:26:59.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:26:59.765 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:26:59.766 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:26:59.767 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:26:59.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:26:59.849 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:26:59.849 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:26:59.850 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:26:59.850 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:26:59.850 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:26:59.851 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:26:59.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:26:59.852 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:26:59.850+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:26:59.858 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:26:59.934 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:26:59.935 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:26:59.952 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:26:59.951+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:00.032 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:00.032 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:27:00.054 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:00.052+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:00.067 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.068 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:00.118 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:00.118 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:27:00.155 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:00.154+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:00.240 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.240 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.241 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:00.241 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:27:00.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:00.292 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:00.290+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:00.325 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:00.326 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:27:00.358 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:00.356+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:00.417 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.417 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:00.458 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:00.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:00.559 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:00.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:00.595 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.595 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:00.661 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:00.659+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:00.734 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.734 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:00.762 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:00.760+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:00.863 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:00.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:00.869 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:00.869 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:00.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:00.964 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:00.963+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:01.005 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.005 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:01.099 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:01.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:01.167 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:01.165+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:01.180 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.181 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:01.274 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:01.272+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:01.354 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.354 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:01.405 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:01.403+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:01.470 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:01.468+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:01.530 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.531 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:01.586 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:01.584+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:01.671 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:01.670+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:01.676 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.676 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:01.772 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:01.771+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:01.857 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:01.857 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:01.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:01.912 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:01.910+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:01.985 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:01.982+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:02.005 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.005 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:02.099 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:02.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:02.176 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:02.174+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:02.181 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.181 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.191 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:02.276 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:02.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:02.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:27:02.357 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.357 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:02.377 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:02.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:02.479 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:02.477+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:02.536 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:02.589 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:02.587+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:02.672 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.672 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:02.682 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:02.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:02.759 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:27:02.760 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-30T06:27:02.760 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T06:27:02.761 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T06:27:02.761 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:27:02.782 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:02.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:02.788 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:27:02.769+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 98443) UID: 0 2023-10-30T06:27:02.788 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:27:02.771+0000 9f64640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:27:02.869 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:02.870 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:02.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:02.919 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:02.918+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:02.983 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:02.982+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:03.084 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.084 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.085 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:03.083+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:03.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:03.186 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:03.184+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:03.281 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.282 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.286 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:03.285+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:03.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:03.331 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:21.155 98448== 2023-10-30T06:27:03.331 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:21.155 98448== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:27:03.341 INFO:tasks.ceph.mon.c.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:27:03.388 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:03.386+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:03.458 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.458 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:03.488 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:03.487+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:03.589 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:03.588+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:03.600 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.601 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:03.690 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:03.689+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:03.748 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.749 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:03.791 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:03.790+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:03.892 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:03.890+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:03.925 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:03.925 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:03.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:03.993 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:03.991+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:04.062 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.063 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:04.112 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:04.111+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:04.194 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:04.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:04.240 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:04.296 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:04.294+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:04.375 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.376 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:04.396 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:04.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:04.497 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:04.496+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:04.553 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.554 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:04.598 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:04.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:04.699 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:04.698+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:04.730 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.730 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:04.800 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:04.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:04.902 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:04.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:04.904 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:04.904 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:04.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:05.003 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:05.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:05.043 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.043 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:05.104 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:05.102+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:05.179 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.179 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:05.204 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:05.203+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:05.305 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:05.304+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:05.354 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.354 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:05.406 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:05.404+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:05.507 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:05.505+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:05.533 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.534 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:05.608 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:05.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:05.681 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.682 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:05.735 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:05.733+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:05.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-30T06:27:05.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-30T06:27:05.809 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:05.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:05.818 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.818 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:05.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:05.912 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:05.910+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:05.995 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:05.996 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.005 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:06.047 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:06.045+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:06.111 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:06.110+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:06.172 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:06.172 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:06.213 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:06.211+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:06.313 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:06.312+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:06.350 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:06.350 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:06.414 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:06.412+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:06.515 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:06.514+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:06.525 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:06.526 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:06.616 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:06.615+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:06.661 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:06.662 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:06.755 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:06.753+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:06.819 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:06.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:06.838 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:06.839 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:06.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:06.933 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:06.931+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:07.016 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.016 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.021 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:07.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:07.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:07.121 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:07.120+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:07.193 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.193 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:07.223 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:07.221+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:07.324 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:07.322+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:07.371 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.372 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:07.465 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:07.463+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:07.533 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:07.531+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:07.546 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.547 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:07.626 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:07.624+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:07.694 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.695 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:07.747 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:07.745+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:07.827 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:07.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:07.871 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:07.872 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:07.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:07.928 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:07.927+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:08.009 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.010 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:08.062 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:08.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:08.130 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:08.129+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:08.186 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.187 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:08.239 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:08.238+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:08.322 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.322 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:08.374 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:08.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:08.434 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:08.432+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:08.501 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.502 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:08.554 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:08.552+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:08.635 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:08.633+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:08.680 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.680 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.690 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:08.736 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:08.735+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:08.838 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:08.836+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:08.862 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:27:08.863 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-30T06:27:08.863 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T06:27:08.864 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T06:27:08.864 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:27:08.869 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:08.869 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:08.877 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:27:08.866+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 94351) UID: 0 2023-10-30T06:27:08.879 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:27:08.868+0000 9f64640 -1 mon.a@0(probing) e1 *** Got Signal Terminated *** 2023-10-30T06:27:08.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:08.938 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:08.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:09.011 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.011 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:09.065 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:09.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:09.140 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:09.138+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:09.148 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.148 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:09.240 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:09.239+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:09.327 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.327 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:09.341 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:09.340+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:09.442 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:09.441+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:09.444 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:11.044 94385== 2023-10-30T06:27:09.444 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:11.044 94385== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:27:09.455 INFO:tasks.ceph.mon.a.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:27:09.463 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.463 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:09.546 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:09.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:09.632 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.633 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:09.682 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:09.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:09.745 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:09.744+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:09.809 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.810 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:09.859 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:09.858+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:09.946 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:09.945+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:09.988 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:09.989 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:09.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:10.047 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:10.046+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:10.126 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.127 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:10.148 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:10.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:10.248 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:10.247+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:10.302 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.302 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:10.352 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:10.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:10.451 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:10.450+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:10.479 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.479 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:10.552 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:10.551+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:10.614 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.614 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:10.653 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:10.651+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:10.751 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.751 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.754 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:10.752+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:10.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:10.855 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:10.853+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:10.888 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:10.888 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:10.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:10.982 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:10.981+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:11.056 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:11.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:11.067 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.068 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:11.161 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:11.160+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:11.243 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.244 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:11.258 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:11.257+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:11.362 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:11.360+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:11.424 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.424 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:11.475 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:11.473+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:11.561 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:11.559+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:11.570 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.570 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:11.661 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:11.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:11.706 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.706 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:11.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-30T06:27:11.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-30T06:27:11.763 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:11.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:11.842 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:11.842 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:11.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:11.863 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:11.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:11.964 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:11.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:12.019 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.019 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:12.064 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:12.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:12.181 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:12.179+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:12.195 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.195 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:12.288 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:12.287+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:12.366 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:12.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:12.372 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.372 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:12.468 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:12.466+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:12.548 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.549 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:12.599 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:12.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:12.669 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:12.668+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:12.724 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.724 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:12.778 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:12.776+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:12.860 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:12.860 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:12.869 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:12.873 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:12.870+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:12.973 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:12.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:13.036 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.037 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:13.073 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:13.072+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:13.173 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.174 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.174 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:13.173+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:13.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:13.275 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:13.273+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:13.315 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.316 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:13.376 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:13.375+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:13.464 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.464 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:13.477 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:13.475+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:13.578 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:13.577+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:13.604 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.605 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:13.678 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:13.678+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:13.739 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.740 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:13.794 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:13.792+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:13.874 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:13.875 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:13.879 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:13.878+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:13.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:13.980 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:13.979+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:14.051 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.051 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:14.082 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:14.081+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:14.183 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:14.181+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:14.185 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.185 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:14.284 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:14.283+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:14.362 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.362 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:14.385 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:14.384+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:14.486 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:14.484+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:14.498 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.498 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:14.593 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:14.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:14.675 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.675 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:14.725 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:14.723+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:14.790 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:14.788+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:14.851 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:14.851 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:14.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:14.902 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:14.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:14.966 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:27:14.967 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-10-30T06:27:14.967 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T06:27:14.967 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2023-10-30T06:27:14.968 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T06:27:14.968 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-30T06:27:14.971 INFO:tasks.ceph.mon.b:Started 2023-10-30T06:27:14.972 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T06:27:14.972 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-30T06:27:14.974 INFO:tasks.ceph.mon.c:Started 2023-10-30T06:27:14.975 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:27:14.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:14.992 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:14.990+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:15.040 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.041 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:15.093 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:15.091+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:15.180 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.180 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:15.193 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:15.191+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:15.294 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:15.292+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:15.413 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:15.411+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:15.496 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:15.494+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:15.497 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.498 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:15.596 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:15.595+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:15.681 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.682 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:15.697 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:15.695+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:15.798 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:15.796+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:15.823 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.823 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:15.899 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:15.898+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:15.963 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:15.963 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:15.972 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:16.016 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:16.015+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:16.101 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:16.100+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:16.112 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.113 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:16.206 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:16.205+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:16.298 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.298 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.303 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:16.301+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:16.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:16.404 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:16.403+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:16.477 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.478 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:16.528 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:16.527+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:16.605 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:16.604+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:16.664 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.664 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:16.706 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:16.704+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:16.803 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.804 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.806 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:16.805+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:16.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:16.907 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:16.906+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:16.942 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:16.942 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:16.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:17.008 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:17.006+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:17.081 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.082 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:17.135 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:17.133+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:17.209 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:17.208+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:17.221 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.222 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:17.311 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:17.310+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:17.401 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.401 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:17.412 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:17.410+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:17.513 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:17.512+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:17.583 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.584 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:17.613 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:17.613+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:17.714 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:17.713+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:17.721 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.721 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:17.815 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:17.813+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:17.858 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:17.859 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:17.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:17.953 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:17.951+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:17.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-30T06:27:18.022 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:18.021+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:18.036 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.036 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:18.119 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:18.119+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:18.214 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.215 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.220 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:18.220+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:18.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:18.321 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:18.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:18.353 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.354 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:18.422 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:18.422+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:18.490 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.491 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:18.545 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:18.543+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:18.624 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:18.622+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:18.628 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.628 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:18.728 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:18.726+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:18.812 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.812 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:18.863 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:18.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:18.926 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:18.924+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:18.989 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:18.989 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:18.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:19.045 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:19.045+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:19.128 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:19.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:19.129 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.129 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:19.229 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:19.227+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:19.307 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.308 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.318 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:19.330 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:19.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:19.432 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:19.430+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:19.446 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.446 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:19.533 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:19.533+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:19.583 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.583 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:19.634 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:19.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:19.735 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:19.734+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:19.837 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:19.836+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:19.898 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:19.899 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:19.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:19.937 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:19.936+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:20.038 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:20.037+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:20.074 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.075 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:20.139 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:20.139+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:20.214 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.214 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:20.240 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:20.239+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:20.342 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:20.341+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:20.400 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.400 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.411 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:20.461 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:20.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:20.543 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.544 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.544 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:20.542+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:20.552 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:20.645 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:20.643+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:20.720 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.720 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:20.745 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:20.743+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:20.846 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:20.845+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:20.899 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:20.900 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:20.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:20.948 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:20.946+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:21.037 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.037 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:21.050 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:21.048+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:21.151 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:21.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:21.177 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.177 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:21.271 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:21.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:21.353 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:21.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:21.359 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.359 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:21.455 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:21.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:21.534 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.535 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:21.589 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:21.587+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:21.659 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:21.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:21.672 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.672 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:21.766 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:21.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:21.849 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:21.849 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:21.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:21.862 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:21.859+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:21.961 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:21.960+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:22.045 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.046 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:22.096 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:22.094+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:22.163 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:22.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:22.230 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.230 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:22.264 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:22.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:22.370 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:22.368+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:22.384 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.384 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:22.478 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:22.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:22.561 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.562 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.567 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:22.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:22.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:22.667 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:22.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:22.741 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.741 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:22.792 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:22.790+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:22.869 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:22.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:22.922 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:22.923 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:22.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:22.973 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:22.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:23.023 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:07.981 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:23.059 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.059 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.068 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:23.070 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:23.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:23.171 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:23.170+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:23.178 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.142 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:23.200 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.200 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:23.263 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.221 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:23.272 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:23.271+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:23.340 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.341 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:23.390 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:23.389+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:23.417 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.381 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:23.474 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:23.473+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:23.535 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.535 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:23.588 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:23.587+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:23.676 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:23.675+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:23.689 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.689 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:23.777 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:23.775+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:23.878 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:23.877+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:23.882 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:23.883 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:23.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:23.979 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:23.978+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:24.038 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.038 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:24.090 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:24.089+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:24.181 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:24.180+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:24.193 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.194 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:24.282 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:24.280+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:24.375 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.375 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.382 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:24.381+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:24.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:24.483 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:24.482+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:24.513 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.514 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:24.585 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:24.583+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:24.651 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.651 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:24.702 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:24.701+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:24.787 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:24.785+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:24.829 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.829 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:24.888 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:24.886+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:24.984 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:24.985 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:24.989 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:24.987+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:24.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:25.090 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:25.088+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:25.178 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:25.178 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:25.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:25.190 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:25.188+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:25.291 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:25.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:25.338 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:25.338 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:25.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:25.431 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:25.429+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:25.493 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:25.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:25.516 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:25.516 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:25.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:25.594 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:25.592+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:25.695 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:25.694+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:25.720 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:25.720 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:25.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:25.796 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:25.795+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:25.859 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:25.860 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:25.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:25.897 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:25.896+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:25.998 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:25.997+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:26.052 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.052 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:26.099 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:26.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:26.200 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:26.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:26.236 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.236 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:26.302 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:26.301+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:26.377 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.378 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:26.431 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:26.429+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:26.505 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:26.503+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:26.526 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.527 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:26.606 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:26.604+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:26.707 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:26.706+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:26.762 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.763 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:26.808 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:26.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:26.928 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:26.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:26.942 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:26.942 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:26.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:27.010 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.080 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.080 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:27.130 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.129+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.211 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.257 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.257 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:27.312 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:27.311+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:27.414 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.412+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.433 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.433 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:27.528 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:27.526+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:27.608 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.608 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:27.618 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.615+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.717 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:27.716+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:27.784 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.784 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:27.819 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:27.817+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:27.919 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:27.918+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:27.958 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:27.959 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:27.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:28.020 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:28.018+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:28.094 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.095 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:28.144 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:28.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:28.222 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:28.221+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:28.270 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.270 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:28.323 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:28.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:28.424 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:28.423+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:28.444 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.444 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:28.525 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:28.524+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:28.582 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.583 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:28.637 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:28.635+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:28.715 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.715 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:28.726 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:28.725+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:28.827 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:28.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:28.889 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:28.889 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:28.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:28.943 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:28.941+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:29.029 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:29.027+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:29.038 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.038 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:29.129 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:29.128+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:29.214 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.214 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:29.230 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:29.229+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:29.332 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:29.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:29.352 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.352 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:29.433 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:29.431+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:29.492 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.492 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:29.533 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:29.532+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:29.625 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.625 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:29.635 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:29.632+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:29.734 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:29.733+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:29.759 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.760 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:29.835 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:29.833+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:29.895 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:29.895 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:29.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:29.950 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:29.948+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:30.031 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.032 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.036 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:30.035+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:30.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:30.138 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:30.136+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:30.211 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.211 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:30.238 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:30.237+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:30.340 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:30.338+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:30.387 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.387 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:30.441 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:30.440+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:30.521 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.521 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:30.542 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:30.541+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:30.648 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:30.647+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:30.658 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.659 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:30.744 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:30.743+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:30.803 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.803 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:30.850 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:30.849+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:30.945 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:30.944+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:30.982 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:30.983 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:30.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:31.047 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:31.045+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:31.119 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.120 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:31.148 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:31.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:31.249 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:31.248+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:31.259 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.259 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:31.357 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:31.355+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:31.436 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.436 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:31.451 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:31.450+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:31.552 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:31.550+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:31.614 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.614 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:31.653 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:31.652+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:31.754 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:31.752+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:31.793 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.794 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:31.855 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:31.853+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:31.957 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:31.956+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:31.970 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:31.971 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:31.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:32.057 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:32.056+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:32.109 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:32.109 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:32.118 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:32.160 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:32.158+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:32.277 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:32.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:32.290 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:32.291 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:32.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:32.361 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:32.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:32.462 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:32.461+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:32.470 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:32.470 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:32.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:32.563 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:32.561+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:32.609 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:32.609 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:32.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:32.663 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:32.662+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:32.764 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:32.763+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:32.784 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:32.784 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:27:32.785 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:32.785 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:32.785 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:32.785 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:32.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:32.865 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:32.864+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:32.870 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:32.871 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:27:32.994 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:32.994 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:27:32.995 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:32.992+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:33.000 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:33.000 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:33.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:33.083 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:33.083 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:27:33.084 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:33.082+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:33.168 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:33.167+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:33.184 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:33.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:27:33.197 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:33.197 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:33.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:33.198 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:33.198 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:33.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:33.283 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:33.284 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:27:33.284 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:33.282+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:33.370 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:33.369+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:33.377 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:33.378 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:27:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:33.382 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:33.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:33.460 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:33.461 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:27:33.472 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:33.470+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:33.559 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:33.559 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:33.560 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:33.560 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:33.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:33.613 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:33.612+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:33.674 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:33.672+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:33.768 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:33.768 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:27:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 200.415628768, 2023-10-30T06:27:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:27:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:27:33.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:33.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:27:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:27:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:27:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:27:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:27:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:27:33.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:27:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:27:33.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:27:33.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:27:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:27:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:27:33.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:27:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:27:33.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:27:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:27:33.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:27:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:27:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:27:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:27:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:27:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:27:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:27:33.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:27:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:27:33.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:27:33.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:27:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 198.094473577, 2023-10-30T06:27:33.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:27:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.791 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:27:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:27:33.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:27:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:27:33.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:27:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:27:33.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:27:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:27:33.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:27:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:27:33.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:27:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:27:33.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:27:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:27:33.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:27:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:27:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:27:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:27:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:27:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:27:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:27:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:27:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:27:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.806 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:33.807 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 198.09022902000001, 2023-10-30T06:27:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:27:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.808 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:27:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:27:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:27:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:27:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:27:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:27:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:27:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:27:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:27:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:27:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:27:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:27:33.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:27:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:27:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:27:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:27:33.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 198.004770595, 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:27:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:27:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:27:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:27:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:27:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:27:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:27:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:27:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:27:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:27:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:27:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:27:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:27:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:27:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:27:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:27:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:27:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:27:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:27:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:27:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:27:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:33.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.73552737899999, 2023-10-30T06:27:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:27:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:27:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:27:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:27:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:27:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:27:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:27:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:27:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:27:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:27:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:27:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:27:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:27:33.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:27:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:27:33.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:27:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:27:33.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.855 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.73469596800001, 2023-10-30T06:27:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:27:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.856 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:27:33.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:33.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:27:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:27:33.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:27:33.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:27:33.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:27:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:27:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:27:33.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:27:33.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:27:33.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:27:33.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:27:33.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:27:33.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:27:33.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:27:33.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:27:33.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:27:33.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:27:33.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:27:33.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.870 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.870 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.734028841, 2023-10-30T06:27:33.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.872 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:27:33.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.873 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:27:33.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:27:33.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:27:33.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:27:33.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:27:33.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:27:33.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:27:33.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:27:33.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:27:33.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:27:33.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:27:33.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:27:33.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:27:33.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:27:33.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:27:33.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:27:33.885 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.885 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.886 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.886 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:33.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.71175477, 2023-10-30T06:27:33.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:27:33.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.887 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.887 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.888 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:27:33.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.888 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:33.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:33.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:27:33.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:27:33.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:27:33.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:27:33.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:27:33.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:27:33.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:27:33.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:27:33.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:27:33.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:27:33.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:27:33.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:27:33.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:27:33.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:27:33.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.711106914, 2023-10-30T06:27:33.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:27:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.903 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:27:33.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:27:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:27:33.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:27:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:27:33.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:27:33.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:27:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:27:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:27:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:27:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 197.710420841, 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:27:33.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:27:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:33.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:27:33.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:27:33.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:27:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:27:33.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:27:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:27:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:27:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:27:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:27:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:27:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:27:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:27:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:27:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:27:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:33.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 196.63953887299999, 2023-10-30T06:27:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:27:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:27:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:27:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:27:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:27:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:27:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:27:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:27:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:27:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:27:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:27:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:27:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:27:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:27:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:27:33.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:27:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:27:33.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:27:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:27:33.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:27:33.948 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.949 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 196.638874907, 2023-10-30T06:27:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:27:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.950 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.951 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:33.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:33.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:27:33.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:27:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:27:33.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:27:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:27:33.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:27:33.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:27:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:27:33.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:27:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:27:33.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:27:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:27:33.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:27:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:27:33.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:27:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:27:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:27:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:27:33.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:27:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:27:33.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 196.61042109799999, 2023-10-30T06:27:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:27:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.966 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:27:33.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:33.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:27:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:27:33.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:27:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:27:33.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:27:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:27:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:27:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:27:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:27:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:27:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:27:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:27:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:27:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:27:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:27:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:27:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:33.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.26865460499999, 2023-10-30T06:27:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:27:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:27:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.983 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:27:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:27:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:27:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:27:33.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:27:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:27:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:27:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:27:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:27:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:27:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:33.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:27:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:27:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:27:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:27:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:27:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:27:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:33.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:33.997 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.26790593499999, 2023-10-30T06:27:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:27:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:33.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:27:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:33.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:27:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:27:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:27:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:27:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:27:34.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:27:34.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:27:34.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:27:34.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:27:34.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:27:34.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:27:34.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:27:34.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:27:34.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:27:34.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:27:34.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:27:34.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:27:34.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:27:34.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:34.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:34.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.26723755200001, 2023-10-30T06:27:34.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:27:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:34.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:34.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:27:34.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.015 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:34.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:34.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:27:34.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:27:34.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:27:34.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:27:34.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:27:34.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:27:34.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:27:34.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:27:34.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:27:34.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:27:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:27:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:27:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:27:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:27:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:27:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:27:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.25738687800001, 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:34.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:34.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:34.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:34.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:27:34.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:34.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:34.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:34.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:27:34.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:27:34.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:27:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:27:34.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:27:34.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:27:34.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:27:34.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:27:34.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:27:34.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:27:34.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:27:34.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:27:34.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:27:34.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:27:34.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:27:34.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:27:34.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:27:34.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:27:34.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.25671306300001, 2023-10-30T06:27:34.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:27:34.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:34.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:34.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:34.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:34.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:34.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:27:34.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:34.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:34.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:27:34.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:27:34.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:27:34.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:27:34.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:27:34.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:27:34.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:27:34.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:27:34.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:27:34.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:27:34.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:27:34.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:27:34.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:27:34.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:27:34.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:27:34.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:27:34.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:34.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.25606468000001, 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:34.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:34.061 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:34.061 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:27:34.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.061 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:34.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:34.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:34.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:27:34.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:27:34.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:27:34.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:27:34.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:27:34.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:27:34.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:27:34.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:27:34.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:27:34.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:27:34.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:27:34.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:27:34.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:27:34.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:27:34.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:27:34.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:27:34.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:27:34.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:27:34.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.074 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.074 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.075 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:34.075 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:34.075 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 189.25194821400001, 2023-10-30T06:27:34.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:34.076 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:27:34.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.077 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:34.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:34.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:34.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:34.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:34.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:34.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:34.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:27:34.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:27:34.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:34.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:27:34.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:27:34.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:34.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:27:34.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:27:34.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:34.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:27:34.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:27:34.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:34.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:27:34.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:27:34.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:34.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:27:34.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:27:34.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:34.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:27:34.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:27:34.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:34.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:27:34.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:27:34.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:34.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:34.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:34.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:27:34.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:27:34.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.090 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:34.090 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:34.091 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:34.093 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.2 pg_upmap_items mapping to [7->4,6->1] 2023-10-30T06:27:34.094 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:27:34.095 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:34.091+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:34.193 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:34.192+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:34.279 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:34.280 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:34.280 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:34.280 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:34.294 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:34.293+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:34.313 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:34.327 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:27:34.327 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T06:27:34.327 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-10-30T06:27:34.370 INFO:tasks.ceph.mon.a:Started 2023-10-30T06:27:34.370 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:27:34.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:34.395 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:34.394+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:34.471 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:34.472 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:34.496 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:34.495+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:34.597 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:34.596+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:34.615 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:34.616 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:34.698 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:34.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:34.754 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:34.755 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:34.755 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:34.813 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:34.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:34.900 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:34.899+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:34.908 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:34.909 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:34.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:35.006 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:35.006+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:35.101 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:35.100+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:35.102 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:35.155 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:35.155 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.156 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:27:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 118.863200822, 2023-10-30T06:27:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:27:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:35.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:27:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:27:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:27:35.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:35.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:35.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:27:35.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:27:35.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:27:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:27:35.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:27:35.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:27:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:27:35.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:27:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:27:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:35.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:35.167 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:35.169 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:35.210 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:35.209+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:35.289 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.252 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.303 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:35.303+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:35.328 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.285 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.349 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:35.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:35.349 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:35.349 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:35.357 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.321 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.359 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:35.405 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:35.404+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:35.445 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.403 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:27:35.488 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:35.488 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:35.488 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:35.489 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:35.489 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:35.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:35.543 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:35.541+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:35.569 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.533 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.607 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:35.607+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:35.624 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:35.625 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:35.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:35.633 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.590 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:35.717 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:35.715+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:35.802 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:35.803 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:35.803 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:35.809 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:35.808+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:35.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:35.884 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:35.885 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:27:35.909 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:35.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:35.973 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:35.973 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:27:35.984 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:35.984 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:35.984 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:35.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:36.010 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:36.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:36.060 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:36.060 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:27:36.149 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:36.150 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.150 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:36.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:36.212 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:36.211+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:36.221 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:36.221 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:36.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:36.221 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:36.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.222 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:27:36.222 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:36.222 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 204.06056942699999, 2023-10-30T06:27:36.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:36.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:27:36.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.224 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:36.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:36.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:27:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:27:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:27:36.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:27:36.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:27:36.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:27:36.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:27:36.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:27:36.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:27:36.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:27:36.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:27:36.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:27:36.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:27:36.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:27:36.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:27:36.234 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.235 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.235 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.235 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:27:36.236 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:36.236 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 201.14739556399999, 2023-10-30T06:27:36.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:27:36.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:36.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:27:36.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:27:36.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:27:36.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:27:36.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:27:36.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:27:36.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:27:36.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:27:36.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:27:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:27:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:27:36.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:27:36.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.248 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:27:36.248 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:36.248 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 200.487217595, 2023-10-30T06:27:36.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:27:36.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.249 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:36.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:27:36.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:27:36.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:27:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:27:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:27:36.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:27:36.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:27:36.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:27:36.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:27:36.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:27:36.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:27:36.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:27:36.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:27:36.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:27:36.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:27:36.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:27:36.259 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.259 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.259 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.260 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:27:36.260 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:36.260 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 200.411794388, 2023-10-30T06:27:36.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:27:36.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.261 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:36.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:36.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:27:36.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:27:36.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:27:36.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:27:36.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:27:36.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:27:36.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:27:36.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:27:36.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:27:36.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:27:36.271 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.272 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.272 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.272 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:27:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 200.14067125700001, 2023-10-30T06:27:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:27:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.274 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:36.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:36.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:27:36.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:27:36.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:27:36.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:27:36.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:27:36.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:27:36.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:27:36.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:27:36.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:27:36.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:27:36.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:27:36.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:27:36.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:27:36.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.284 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:27:36.285 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:36.285 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 200.11039154700001, 2023-10-30T06:27:36.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:27:36.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.286 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:36.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:27:36.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:27:36.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:27:36.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:27:36.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:27:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:27:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:27:36.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:27:36.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:27:36.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:27:36.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:27:36.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:27:36.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:27:36.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:27:36.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:27:36.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:27:36.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.298 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.299 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:27:36.300 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:36.300 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.73648046299999, 2023-10-30T06:27:36.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:27:36.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.301 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:36.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:36.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:27:36.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:27:36.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:27:36.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:27:36.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:27:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:27:36.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:27:36.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:27:36.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:27:36.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:27:36.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:27:36.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:27:36.310 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.310 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.311 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:27:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.731981762, 2023-10-30T06:27:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:27:36.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:36.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:36.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:27:36.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:27:36.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:27:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:27:36.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:27:36.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:27:36.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:27:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:27:36.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:27:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:27:36.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:27:36.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:27:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.44037782300001, 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.325 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:36.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:36.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:27:36.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:27:36.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:27:36.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:27:36.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:27:36.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:27:36.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:27:36.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:27:36.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:27:36.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:27:36.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:27:36.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:27:36.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.335 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.336 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.336 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:27:36.336 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:36.337 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.43444050599999, 2023-10-30T06:27:36.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:27:36.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.337 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:36.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:27:36.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:27:36.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:27:36.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:27:36.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:27:36.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:27:36.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:27:36.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:27:36.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:27:36.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:27:36.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:27:36.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:27:36.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:27:36.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:27:36.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:27:36.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:27:36.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.348 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.348 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.43314149099999, 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.349 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:36.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:36.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:27:36.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:27:36.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:27:36.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:27:36.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:27:36.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:27:36.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:27:36.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:27:36.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:27:36.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:27:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:27:36.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:27:36.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:27:36.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.360 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:27:36.361 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:36.361 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.33015268299999, 2023-10-30T06:27:36.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:27:36.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:36.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:36.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:27:36.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:27:36.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:27:36.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:27:36.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:27:36.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:27:36.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:27:36.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:27:36.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:27:36.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:27:36.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:27:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:27:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:27:36.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:27:36.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.372 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.374 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:27:36.374 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:36.375 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.058249416, 2023-10-30T06:27:36.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:27:36.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:36.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:36.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:27:36.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:27:36.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:27:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:27:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:27:36.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:27:36.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:27:36.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:27:36.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:27:36.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:27:36.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:27:36.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:27:36.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.389 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:27:36.389 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:36.389 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.05075122400001, 2023-10-30T06:27:36.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:27:36.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.390 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:36.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:27:36.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:27:36.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:27:36.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:27:36.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:27:36.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:27:36.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:27:36.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:27:36.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:27:36.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:27:36.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:27:36.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:27:36.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:27:36.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:27:36.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:27:36.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:27:36.405 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.405 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.405 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.406 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:27:36.406 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:36.407 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.05009001900001, 2023-10-30T06:27:36.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:27:36.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.407 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:36.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:27:36.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:27:36.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:27:36.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:27:36.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:27:36.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:27:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:27:36.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:27:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:27:36.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:27:36.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:27:36.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:27:36.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:27:36.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:27:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:27:36.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:27:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:27:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.037504181, 2023-10-30T06:27:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:27:36.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:36.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:36.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:27:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:27:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:27:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:27:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:27:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:27:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:27:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:27:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:27:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:27:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:27:36.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:27:36.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:27:36.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:27:36.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:27:36.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:27:36.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.430 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.431 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:27:36.431 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:36.431 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.03141196799999, 2023-10-30T06:27:36.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:27:36.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.432 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:36.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:36.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:27:36.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:27:36.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:27:36.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:27:36.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:27:36.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:27:36.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:27:36.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:27:36.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:27:36.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:27:36.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:27:36.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:27:36.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:27:36.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:27:36.442 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.442 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:36.443 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 199.03041662699999, 2023-10-30T06:27:36.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:27:36.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.444 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:27:36.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:27:36.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:27:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:27:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:27:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:27:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:27:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:27:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:27:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:27:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:27:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:27:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:27:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:27:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.454 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:27:36.455 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 191.707744056, 2023-10-30T06:27:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:27:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.456 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:36.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:27:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:27:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:27:36.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:27:36.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:27:36.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:27:36.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:27:36.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:27:36.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:27:36.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:27:36.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:27:36.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:27:36.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:27:36.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:27:36.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.467 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 191.674302571, 2023-10-30T06:27:36.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:27:36.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:36.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:36.469 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:36.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:36.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:36.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:36.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:36.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:36.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:36.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:27:36.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:27:36.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:36.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:27:36.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:27:36.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:36.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:27:36.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:27:36.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:36.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:27:36.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:27:36.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:36.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:27:36.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:27:36.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:36.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:27:36.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:27:36.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:36.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:36.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:36.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:27:36.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:27:36.480 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.481 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:36.482 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:36.482 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:36.484 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:36.484 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.485 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:36.486 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:36.487 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:36.488 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.490 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:36.484+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:36.525 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:36.525 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:27:36.585 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:36.583+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:36.616 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:36.616 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:36.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:36.685 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:36.685+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:36.786 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:36.785+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:36.810 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:36.810 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:36.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:36.887 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:36.887+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:36.986 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:36.986 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:36.988 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:36.986+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:36.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:37.088 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:37.088+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:37.161 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.161 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:37.213 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:37.212+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:37.289 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:37.288+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:37.299 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.299 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.309 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:37.359 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:27:37.393 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:37.392+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:37.477 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.477 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:37.527 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:37.526+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:37.591 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:37.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:37.613 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.614 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:37.692 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:37.690+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:37.790 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.790 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.792 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:37.791+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:37.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:37.893 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:37.892+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:37.967 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:37.967 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:37.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:38.020 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:38.019+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:38.096 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:38.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:38.104 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:38.104 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:38.113 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:38.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:38.197 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:38.196+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:38.292 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:38.292 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:38.297 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:38.297+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:38.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:38.398 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:38.397+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:38.469 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:38.469 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:38.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:38.499 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:38.498+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:38.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:27:38.622 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:38.622+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:38.638 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:38.638 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:38.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:38.735 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:38.734+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:38.802 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:38.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:38.807 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:38.807 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:27:38.846 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:38.847 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:38.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:38.900 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:38.900 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:27:38.902 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:38.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:38.910 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:38.988 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:38.988 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.003 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:39.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:39.028 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:39.028 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:39.029 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:39.029 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:39.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:39.075 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:39.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.094 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:27:39.095 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:27:39.095 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2023-10-30T06:27:39.095 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:27:39.096 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:27:39.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2023-10-30T06:27:39.104 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:39.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:39.165 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:39.166 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:39.166 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:39.166 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:39.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:39.176 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:39.177 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.204 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:39.203+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:39.228 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:39.228 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2023-10-30T06:27:39.228 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:39.277 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:39.277 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.305 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:39.304+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:39.306 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:39.306 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:39.307 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:39.307 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:39.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:39.361 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:39.361 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:27:39.406 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:39.405+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:39.456 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:39.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:27:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 123.164952754, 2023-10-30T06:27:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:27:39.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:27:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:27:39.459 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:39.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:39.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:39.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:27:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:27:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:39.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:27:39.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:27:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:27:39.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:27:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:27:39.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:27:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:27:39.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:27:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:39.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:39.469 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:39.469 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:39.469 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:27:39.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:39.520 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:39.518+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:39.602 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:39.602 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:39.603 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:39.603 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:39.608 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:39.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:39.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:39.708 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:39.707+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:39.792 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:39.793 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:39.793 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:39.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:39.793 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:39.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:39.809 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:39.808+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:39.910 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:39.909+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:39.928 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:39.928 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:39.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:39.928 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:39.929 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:39.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:40.011 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:40.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:40.063 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:40.063 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:40.063 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:40.064 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:40.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:40.112 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:40.111+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:40.213 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:40.212+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:40.244 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:40.245 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:40.245 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:40.245 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:40.245 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:40.250 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:25.214 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:40.297 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:25.261 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.313 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:25.277 106787== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.338 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:40.336+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:40.348 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:25.306 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.352 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:25.310 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.365 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:25.322 98850== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:40.415 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:40.414+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:40.478 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:40.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 208.309630825, 2023-10-30T06:27:40.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:27:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:27:40.480 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:27:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:27:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:27:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:27:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:27:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:27:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:27:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:27:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:27:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:27:40.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:27:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:27:40.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:27:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:27:40.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:27:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 205.396456962, 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:40.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:40.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:27:40.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:27:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:27:40.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:27:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:27:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:27:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:27:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:27:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:27:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:27:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:27:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:27:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:27:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:27:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 204.73627899300001, 2023-10-30T06:27:40.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:27:40.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:27:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:27:40.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:27:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:27:40.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:27:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:27:40.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:27:40.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:27:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:27:40.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:27:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:27:40.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:27:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:27:40.515 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.516 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:27:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 204.66085578600001, 2023-10-30T06:27:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:27:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:27:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:27:40.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:27:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:27:40.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:27:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:27:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:27:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:27:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:27:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:27:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:27:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 204.38973265499999, 2023-10-30T06:27:40.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:27:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:27:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:27:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:27:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:27:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:27:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:27:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:27:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:27:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:27:40.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:27:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:27:40.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:27:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:27:40.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:27:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:27:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 204.35945294499999, 2023-10-30T06:27:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:27:40.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:27:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:27:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:27:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:27:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:27:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:27:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:27:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:27:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:27:40.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:27:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:27:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:27:40.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:27:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.985541861, 2023-10-30T06:27:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:27:40.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:27:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:27:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:27:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:27:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:27:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:27:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:27:40.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:27:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:27:40.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:27:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:27:40.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:27:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:27:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.98104316000001, 2023-10-30T06:27:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:27:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:27:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:27:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:27:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:27:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:27:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:27:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:27:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:27:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:27:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:27:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:27:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:27:40.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.575 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.68943922099999, 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.576 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:27:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:27:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:27:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:27:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:27:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:27:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:27:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:27:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:27:40.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:27:40.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:27:40.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:27:40.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.586 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.586 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.683501904, 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:27:40.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:40.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:40.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:27:40.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:27:40.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:27:40.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:27:40.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:27:40.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:27:40.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:27:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:27:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:27:40.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:27:40.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:27:40.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:27:40.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:27:40.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:27:40.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.598 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.599 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:27:40.599 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:40.599 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.682202889, 2023-10-30T06:27:40.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:27:40.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.600 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:40.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:27:40.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:27:40.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:27:40.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:27:40.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:27:40.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:27:40.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:27:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:27:40.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:27:40.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:27:40.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:27:40.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:27:40.609 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.609 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:40.610 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.579214081, 2023-10-30T06:27:40.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:27:40.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.611 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:40.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:40.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:27:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:27:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:27:40.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:27:40.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:27:40.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:27:40.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:27:40.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:27:40.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:27:40.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:27:40.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.622 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:27:40.622 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:40.622 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.307310814, 2023-10-30T06:27:40.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:27:40.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:40.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:40.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:27:40.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:27:40.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:27:40.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:27:40.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:27:40.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:27:40.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:27:40.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:27:40.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:27:40.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:27:40.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:27:40.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:27:40.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:27:40.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:27:40.632 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.633 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:27:40.634 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:40.634 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.29981262199999, 2023-10-30T06:27:40.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:27:40.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:40.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:27:40.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:27:40.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:27:40.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:27:40.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:27:40.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:27:40.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:27:40.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:27:40.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:27:40.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:27:40.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.644 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.645 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:27:40.645 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:40.645 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.29915141699999, 2023-10-30T06:27:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:27:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:27:40.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:27:40.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:27:40.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:27:40.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:27:40.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:27:40.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:27:40.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:27:40.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.656 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.28656557900001, 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.658 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:40.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:40.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:27:40.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:27:40.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:27:40.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:27:40.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:27:40.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:27:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:27:40.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:27:40.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:27:40.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:27:40.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:27:40.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:27:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:27:40.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.668 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.668 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.280473366, 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.669 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:27:40.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:27:40.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:27:40.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:27:40.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:27:40.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:27:40.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:27:40.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:27:40.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:27:40.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:27:40.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:27:40.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:27:40.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.679 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 203.279478025, 2023-10-30T06:27:40.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:27:40.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.681 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:40.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:27:40.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:27:40.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:27:40.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:27:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:27:40.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:27:40.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:27:40.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:27:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:27:40.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:27:40.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:27:40.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:27:40.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.691 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.691 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:27:40.692 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:40.692 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 195.956805454, 2023-10-30T06:27:40.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:27:40.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:40.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:27:40.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:27:40.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:27:40.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:27:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:27:40.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:27:40.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:27:40.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:27:40.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:27:40.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:27:40.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:27:40.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:27:40.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:27:40.702 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.702 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.703 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:27:40.703 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:40.704 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 195.92336396900001, 2023-10-30T06:27:40.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:27:40.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:40.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:40.704 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:40.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:40.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:40.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:40.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:27:40.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:27:40.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:40.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:27:40.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:27:40.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:40.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:27:40.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:27:40.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:40.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:27:40.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:27:40.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:27:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:27:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:40.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:27:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:27:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:40.715 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:40.715 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:40.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:40.718 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:40.716+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:40.818 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:40.817+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:40.885 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:40.886 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:40.886 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:40.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:40.937 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:40.935+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:40.997 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:27:41.054 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:41.054+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:41.068 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:41.069 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:41.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:41.120 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:41.120+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:41.204 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:41.205 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:41.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:41.205 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:41.205 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:41.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:41.221 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:41.220+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:41.322 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:41.321+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:41.339 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:41.339 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:41.339 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:41.340 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:41.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:41.423 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:41.422+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:41.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:41.476 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:41.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:27:41.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:41.524 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:41.523+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:41.625 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:41.624+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:41.654 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:27:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 124.68239944299999, 2023-10-30T06:27:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:27:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:27:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:27:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:27:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:27:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:27:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:27:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:27:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:27:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:27:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:27:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:27:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:27:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:41.667 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:41.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:41.710 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:41.711 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:27:41.725 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:41.724+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:41.798 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:41.799 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:27:41.799 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:41.800 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:41.809 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:41.827 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:41.825+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:41.893 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:41.893 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:27:41.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:41.927 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:41.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:41.983 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:41.983 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:27:42.029 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:42.028+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:42.045 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.045 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:42.077 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:42.077 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:27:42.129 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:42.129+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:42.166 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:27:42.213 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.213 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:42.251 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:42.252 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:27:42.253 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:42.251+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:42.331 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:42.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:42.334 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:42.335 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:27:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.383 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:42.432 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:42.431+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:42.533 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:42.532+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:42.575 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.576 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:42.635 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:42.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:42.653 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:42.722 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.722 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:42.736 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:42.735+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:42.837 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:42.836+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:42.860 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:42.860 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:42.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:42.938 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:42.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:43.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:27:43.015 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.571 108046== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:43.039 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:43.038+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:43.082 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.082 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:43.140 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:43.139+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:43.224 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.224 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:43.241 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:43.240+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:43.266 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.821 108046== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:43.343 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:43.342+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:43.360 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.361 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:43.454 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:43.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:43.537 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.538 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.544 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:43.544+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:43.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:43.645 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:43.645+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:43.716 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.717 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:43.767 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:43.765+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:43.847 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:43.845+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:43.867 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:43.867 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:43.876 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:43.948 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:43.946+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:44.003 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.004 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:44.057 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:44.056+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:44.118 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:27:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.149 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.150 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:44.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:44.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:44.238 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:27:44.238 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:27:44.238 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2023-10-30T06:27:44.239 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:27:44.239 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:27:44.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2023-10-30T06:27:44.251 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:44.250+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:44.324 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.325 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:44.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:27:44.353 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:44.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:44.367 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:44.368 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "" 2023-10-30T06:27:44.368 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:44.453 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:44.451+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:44.508 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.509 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:44.553 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:44.552+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:44.583 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:44.584 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:27:44.645 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.645 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.654 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:44.652+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:44.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:44.676 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:44.677 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:27:44.755 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:44.753+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:44.763 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:44.764 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:27:44.781 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.781 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:44.856 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:44.854+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:44.864 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:44.865 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:27:44.917 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:44.917 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:44.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:44.950 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:44.950 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:27:44.967 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:44.966+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:45.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:45.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:27:45.062 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:45.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:45.129 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:27:45.130 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:45.130 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:45.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:45.131 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:45.131 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:45.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:45.215 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:27:45.216 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:45.213+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:45.268 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:45.266+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:45.322 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:45.323 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:45.323 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:45.323 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:45.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:45.361 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:45.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:45.462 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:45.461+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:45.498 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:45.499 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:45.499 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:45.499 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:45.499 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:45.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:45.567 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:45.566+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:45.664 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:45.663+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:45.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:45.679 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:45.679 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:45.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:45.680 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:45.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.680 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:27:45.680 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:45.680 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 128.70700857, 2023-10-30T06:27:45.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:27:45.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:45.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:45.681 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:45.681 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:45.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:45.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:45.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:45.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:27:45.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:27:45.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:45.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:27:45.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:27:45.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:45.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:27:45.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:27:45.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:27:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:27:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:45.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:45.691 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:45.691 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:45.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:45.766 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:45.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:45.844 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:45.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:45.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:45.866 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:45.865+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:45.967 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:45.966+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:46.003 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:46.003 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:46.004 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:46.004 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:46.004 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:46.012 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:46.068 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:46.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:46.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:27:46.169 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:46.168+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:46.202 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:46.202 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:46.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:46.202 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:46.203 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:46.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:46.271 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:46.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:46.359 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:46.359 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:46.360 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:46.360 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:46.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:46.372 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:46.370+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:46.473 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:46.471+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:46.528 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.084 108046== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:46.541 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:46.542 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:46.542 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:46.550 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:46.583 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.138 108046== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:46.583 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:46.581+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:46.596 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":26,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:46.675 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:46.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:46.776 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:46.774+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:46.804 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:46.804 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:46.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:46.805 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:46.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.805 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:27:46.805 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:46.806 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.446672337, 2023-10-30T06:27:46.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:27:46.806 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.806 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.806 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.807 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.807 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:27:46.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.807 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:46.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:46.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:27:46.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:27:46.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:27:46.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:27:46.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:27:46.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:27:46.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:27:46.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:27:46.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:27:46.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:27:46.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:27:46.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:27:46.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:27:46.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:27:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:27:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:27:46.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:27:46.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:27:46.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:27:46.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:27:46.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:27:46.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.823 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.823 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:46.823 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.12551714599999, 2023-10-30T06:27:46.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:27:46.824 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.824 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.824 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.825 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:27:46.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.825 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:46.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:46.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:27:46.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:27:46.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:27:46.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:27:46.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:27:46.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:27:46.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:27:46.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:27:46.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:27:46.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:27:46.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:27:46.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:27:46.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:27:46.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:27:46.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:27:46.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:27:46.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:27:46.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:27:46.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.838 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.839 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.121272589, 2023-10-30T06:27:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:27:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:27:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:27:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:27:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:27:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:27:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:27:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:27:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:27:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:27:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:27:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:27:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:27:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:27:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:27:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:27:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:27:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:27:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:27:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:27:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.03581416399999, 2023-10-30T06:27:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:27:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:27:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:27:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:27:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:27:46.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:27:46.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:27:46.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:27:46.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:27:46.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:27:46.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:27:46.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:27:46.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:27:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:27:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:27:46.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:27:46.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:27:46.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:27:46.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:27:46.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:27:46.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:46.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.76657094800001, 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:27:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:46.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:46.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:27:46.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:27:46.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:27:46.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:27:46.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:27:46.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:27:46.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:27:46.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:27:46.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:27:46.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:27:46.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:27:46.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:27:46.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:27:46.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:27:46.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:27:46.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:27:46.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.889 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.890 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:46.890 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.765739537, 2023-10-30T06:27:46.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:27:46.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.890 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.891 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.891 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:27:46.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:46.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:27:46.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:27:46.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:27:46.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:27:46.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:27:46.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:27:46.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:27:46.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:27:46.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:27:46.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:27:46.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:27:46.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:27:46.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:27:46.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:27:46.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:27:46.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:27:46.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:27:46.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:27:46.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:46.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.76507240999999, 2023-10-30T06:27:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:27:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:27:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:27:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:27:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:27:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:27:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:27:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:27:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:27:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:27:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:27:46.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:27:46.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:27:46.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:27:46.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:27:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:27:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:27:46.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:27:46.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:27:46.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:27:46.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:46.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.74279833899999, 2023-10-30T06:27:46.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:27:46.921 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:27:46.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:46.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:27:46.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:27:46.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:27:46.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:27:46.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:27:46.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:27:46.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:27:46.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:27:46.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:27:46.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:27:46.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:27:46.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:27:46.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:27:46.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:27:46.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:27:46.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:27:46.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.936 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:46.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.74215048299999, 2023-10-30T06:27:46.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:27:46.937 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:46.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:46.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:27:46.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:27:46.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:27:46.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:27:46.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:27:46.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:27:46.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:27:46.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:27:46.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:27:46.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:27:46.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:27:46.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:27:46.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:27:46.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:27:46.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:27:46.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:27:46.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:46.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:46.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 210.74146440999999, 2023-10-30T06:27:46.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:27:46.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:27:46.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:46.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:27:46.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:27:46.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:27:46.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:27:46.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:27:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:27:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:27:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:27:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:27:46.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:27:46.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:27:46.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:27:46.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:27:46.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:27:46.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:27:46.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:27:46.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:27:46.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:27:46.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:46.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:46.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 209.67058244200001, 2023-10-30T06:27:46.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:27:46.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:27:46.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:46.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:27:46.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:27:46.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:27:46.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:27:46.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:27:46.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:27:46.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:27:46.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:27:46.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:27:46.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:27:46.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:27:46.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:27:46.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:27:46.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:27:46.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:27:46.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:27:46.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:27:46.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:27:46.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 209.66991847599999, 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:46.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:27:46.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.985 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:46.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:46.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:46.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:46.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:46.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:27:46.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:27:46.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:46.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:27:46.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:27:46.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:46.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:27:46.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:27:46.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:46.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:27:46.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:27:46.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:46.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:27:46.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:27:46.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:46.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:27:46.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:27:46.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:46.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:27:46.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:27:46.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:46.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:27:46.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:27:46.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:46.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:46.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:46.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:46.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:46.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:46.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 209.64146466700001, 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:46.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:27:47.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:47.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:27:47.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:27:47.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:27:47.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:27:47.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:27:47.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:27:47.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:27:47.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:27:47.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:27:47.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:27:47.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:27:47.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:27:47.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:27:47.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:27:47.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:27:47.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:27:47.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:27:47.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:27:47.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.29969817400001, 2023-10-30T06:27:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:27:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:27:47.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.015 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:47.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:27:47.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:27:47.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:27:47.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:27:47.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:27:47.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:27:47.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:27:47.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:27:47.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:27:47.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:27:47.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:27:47.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:27:47.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:27:47.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:27:47.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:27:47.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:27:47.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.029 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.029 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:47.029 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.29894950400001, 2023-10-30T06:27:47.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:27:47.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.030 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.030 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:47.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:47.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:27:47.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:27:47.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:27:47.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:27:47.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:27:47.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:27:47.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:27:47.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:27:47.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:27:47.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:27:47.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:27:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:27:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:27:47.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:27:47.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:27:47.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:27:47.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:27:47.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:27:47.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.044 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.044 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.298281121, 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.046 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.046 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:27:47.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.046 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:47.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:47.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:27:47.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:27:47.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:27:47.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:27:47.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:27:47.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:27:47.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:27:47.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:27:47.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:27:47.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:27:47.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:27:47.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:27:47.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:27:47.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:27:47.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:27:47.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:27:47.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.062 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.062 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.063 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.063 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:47.063 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.288430447, 2023-10-30T06:27:47.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:27:47.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.065 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:47.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:47.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:27:47.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:27:47.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:27:47.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:27:47.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:27:47.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:27:47.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:27:47.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:27:47.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:27:47.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:27:47.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:27:47.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:27:47.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:27:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:27:47.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:27:47.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:27:47.077 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.077 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.078 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.078 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.078 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.287756632, 2023-10-30T06:27:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:27:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:47.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:47.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:27:47.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:27:47.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:27:47.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:27:47.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:27:47.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:27:47.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:27:47.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:27:47.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:27:47.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:27:47.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:27:47.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:27:47.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:27:47.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:27:47.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:27:47.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:27:47.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.093 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.093 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.094 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.094 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:47.094 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.287108249, 2023-10-30T06:27:47.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:27:47.094 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.095 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.095 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.095 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:47.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:47.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:27:47.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:27:47.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:27:47.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:27:47.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:27:47.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:27:47.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:27:47.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:27:47.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:27:47.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:27:47.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:27:47.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:27:47.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:27:47.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:27:47.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:27:47.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:27:47.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:27:47.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:27:47.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.108 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:47.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:47.110 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 202.282991783, 2023-10-30T06:27:47.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:27:47.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:47.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:47.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:47.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:47.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:47.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:27:47.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.111 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:47.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:47.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:47.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:47.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:47.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:47.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:47.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:27:47.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:27:47.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:47.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:27:47.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:27:47.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:47.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:27:47.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:27:47.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:47.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:27:47.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:27:47.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:47.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:27:47.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:27:47.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:47.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:27:47.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:27:47.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:47.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:47.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:47.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:27:47.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:27:47.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.124 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.124 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:47.124 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:47.125 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:47.127 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.406 108046== Warning: unimplemented fcntl command: 1036 2023-10-30T06:27:47.127 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:47.125+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:47.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:27:47.227 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:47.226+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:47.321 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:47.322 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.328 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:47.326+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:47.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:47.429 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:47.428+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:47.482 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:47.483 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:47.531 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:47.529+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:47.630 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:47.629+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:47.635 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:47.635 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:47.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:47.635 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:47.636 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:47.731 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:47.730+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:47.776 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:47.777 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:47.777 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:47.832 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:47.831+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:47.881 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:47.881 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:27:47.956 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:47.955+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:47.970 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:27:47.971 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:47.971 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:47.972 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:47.972 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:47.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:48.034 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:48.033+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:48.057 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:48.058 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.136 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:48.134+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:48.173 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:48.174 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.209 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:48.209 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:48.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:48.268 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:48.268 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.269 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:48.267+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:48.337 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:48.335+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:48.351 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:48.352 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:48.354 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:48.354 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:48.437 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:48.436+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:48.441 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:48.442 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.529 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:48.530 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:27:48.539 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:48.538+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:48.553 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:48.553 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:48.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:48.639 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:48.638+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:48.735 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:48.735 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:48.741 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:48.738+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:48.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:48.752 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:27:48.841 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:48.839+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:48.910 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:48.910 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:48.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:48.965 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:48.963+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:49.042 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:49.040+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:49.047 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.047 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.057 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:49.143 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:49.141+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:49.225 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.226 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:49.244 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:49.242+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:49.344 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:49.343+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:49.360 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.360 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:49.377 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:27:49.377 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:27:49.377 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:27:49.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2023-10-30T06:27:49.446 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:49.444+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:49.496 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.496 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:49.550 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:49.549+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:49.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:27:49.637 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.637 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:49.647 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:49.645+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:49.748 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:49.747+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:49.815 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.816 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:49.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:49.849 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:49.848+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:49.950 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:49.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:49.999 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:49.999 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:50.056 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:50.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:50.153 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:50.152+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:50.155 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:50.156 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:50.254 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:50.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:50.258 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:27:50.238+0000 ff6f640 -1 mon.a@0(leader).osd e64 definitely_dead 0 2023-10-30T06:27:50.325 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:27:50.327 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:50.327 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:50.339 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:27:50.339 INFO:tasks.mon_recovery:stopping all monitors 2023-10-30T06:27:50.340 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T06:27:50.340 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:27:50.362 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:27:50.347+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 98848) UID: 0 2023-10-30T06:27:50.362 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:27:50.349+0000 9f64640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:27:50.382 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:50.381+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:50.455 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:50.454+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:50.528 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:50.528 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:50.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:27:50.556 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:50.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:50.680 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:50.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:50.695 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:50.695 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:50.741 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:27:50.730+0000 ff6f640 -1 mon.a@0(leader).osd e64 definitely_dead 0 2023-10-30T06:27:50.758 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:50.756+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:50.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:27:50.858 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:50.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:50.872 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:27:50.872 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:27:50.909 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:27:50.909 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:27:50.911 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:35.868 98850== 2023-10-30T06:27:50.911 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:35.868 98850== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:27:50.920 INFO:tasks.ceph.mon.b.smithi112.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:27:50.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:50.968 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:50.966+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:50.974 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:27:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.060 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:51.058+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:51.067 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:51.067 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:51.068 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:51.068 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:51.068 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:51.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:51.077 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:27:51.078 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.161 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:51.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:51.164 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:27:51.164 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.247 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:27:51.248 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.261 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:51.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:51.282 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:51.283 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 217.94562290900001, 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:27:51.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:27:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:27:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:27:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:27:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:27:51.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:27:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:27:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:27:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:27:51.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:27:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:27:51.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:27:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:27:51.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:27:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:27:51.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:27:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:27:51.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:27:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:27:51.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:27:51.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:27:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:27:51.302 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.303 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.62446771800001, 2023-10-30T06:27:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:27:51.304 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.305 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:27:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.306 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:27:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:27:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:27:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:27:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:27:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:27:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:27:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:27:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:27:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:27:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:27:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:27:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:27:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:27:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:27:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:27:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:27:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:27:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:51.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.62022316100001, 2023-10-30T06:27:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:27:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:27:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:27:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:27:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:27:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:27:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:27:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:27:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:27:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:27:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:27:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:27:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:27:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:27:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:27:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:27:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:27:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:27:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:27:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:27:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.534764736, 2023-10-30T06:27:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:27:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.337 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:27:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:27:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:27:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:27:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:27:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:27:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:27:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:27:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:27:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:27:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:27:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:27:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:27:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:27:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:27:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:27:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:27:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:27:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:27:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:27:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:27:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:51.353 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.26552151999999, 2023-10-30T06:27:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:27:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:27:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:27:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:27:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:27:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:27:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:27:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:27:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:27:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:27:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:27:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:27:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:27:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:27:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:27:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:27:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:27:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:27:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:27:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:27:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.26469010900001, 2023-10-30T06:27:51.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:27:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.370 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:27:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:27:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:27:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:27:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:27:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:27:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:27:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:27:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:27:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:27:51.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:27:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:27:51.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:27:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:27:51.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:27:51.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:27:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:27:51.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:27:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:27:51.383 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.384 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.264022982, 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.385 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:27:51.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:27:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:27:51.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:27:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:27:51.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:27:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:27:51.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:27:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:27:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:27:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:27:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:27:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:27:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:27:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:27:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:27:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:27:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:27:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:27:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:27:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:27:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.399 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.400 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.241748911, 2023-10-30T06:27:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:27:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.401 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:27:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.402 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:27:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:27:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:27:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:27:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:27:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:27:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:27:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:27:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:27:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:27:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:27:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:27:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:27:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:27:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:27:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:27:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:27:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:27:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.241101055, 2023-10-30T06:27:51.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:27:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.417 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:27:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:51.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:27:51.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:27:51.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:27:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:27:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:27:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:27:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:27:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:27:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:27:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:27:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:27:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:27:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:27:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:27:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:27:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:27:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:27:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:27:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.240414982, 2023-10-30T06:27:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:27:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:27:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:27:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:27:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:27:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:27:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:27:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:27:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:27:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:27:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:27:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:27:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:27:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:27:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:27:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:27:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:27:51.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:27:51.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:27:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:27:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:51.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 214.169533014, 2023-10-30T06:27:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:27:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:27:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:27:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:27:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:27:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:27:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:27:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:27:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:27:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:27:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:27:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:27:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:27:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:27:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:27:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:27:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:27:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:27:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:27:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 214.168869048, 2023-10-30T06:27:51.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:27:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:27:51.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:27:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:27:51.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:27:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:27:51.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:27:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:27:51.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:27:51.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:27:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:27:51.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:27:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:27:51.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:27:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:27:51.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:27:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:27:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:27:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:27:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:51.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 214.14041523899999, 2023-10-30T06:27:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:27:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:27:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:27:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:27:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:27:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:27:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:27:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:27:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:27:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:27:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:27:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:27:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:27:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:27:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:27:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:27:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:27:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:27:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:27:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:27:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:27:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:51.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.798648746, 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:27:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:27:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:27:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:27:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:27:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:27:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:27:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:27:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:27:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:27:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:27:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:27:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:27:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:27:51.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:27:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:27:51.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:27:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.79790007599999, 2023-10-30T06:27:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:27:51.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:27:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:51.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:27:51.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:27:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:27:51.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:27:51.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:27:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:27:51.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:27:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:27:51.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:27:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:27:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:27:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:27:51.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:27:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:27:51.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:27:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:27:51.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:27:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:27:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.79723169299999, 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:27:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:27:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:27:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:27:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:27:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:27:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:27:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:27:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:27:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:27:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:27:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:27:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:27:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:27:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:27:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:27:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:27:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:27:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:27:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:51.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.78738101900001, 2023-10-30T06:27:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:27:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:27:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:27:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:27:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:27:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:27:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:27:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:27:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:27:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:27:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:27:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:27:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:27:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:27:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:27:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:27:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:27:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:27:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:27:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:27:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:27:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:27:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.78670720400001, 2023-10-30T06:27:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:27:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:27:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:27:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:27:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:27:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:27:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:27:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:27:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:27:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:27:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:27:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:27:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:27:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:27:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:27:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:27:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:27:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:27:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:27:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:27:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:27:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:27:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:27:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:27:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.78605882100001, 2023-10-30T06:27:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:27:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:27:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:27:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:27:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:27:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:27:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:27:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:27:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:27:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:27:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:27:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:27:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:27:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:27:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:27:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:27:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:27:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:27:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:27:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:27:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:27:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:27:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 206.78194235500001, 2023-10-30T06:27:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:27:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:27:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:27:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:27:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:27:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:27:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:27:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:27:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:27:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:27:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:27:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:27:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:27:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:27:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:27:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:27:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:27:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:27:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:27:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:27:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:51.607 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:51.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:51.610 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:27:51.610 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.610 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:27:51.611 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.611 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:27:51.612 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:27:51.613 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:51.608+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:51.710 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:51.710+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:51.778 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:51.779 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:51.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:51.831 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:51.829+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:51.913 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:51.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:51.916 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:51.917 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:51.917 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:51.917 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:51.917 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:51.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:52.014 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:52.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:52.094 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:52.094 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:52.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:52.094 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:52.095 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:52.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:52.145 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:52.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:52.216 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:52.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:52.273 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:52.274 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:52.274 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:52.274 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:52.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:52.316 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:52.315+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:52.410 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:52.410 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:52.410 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:52.411 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:52.411 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:52.416 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:52.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:52.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:52.518 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:52.516+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:52.548 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:52.548 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:27:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:52.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 136.256306228, 2023-10-30T06:27:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:27:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:27:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:52.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:27:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:27:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:52.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:52.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:27:52.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:27:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:27:52.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:27:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:27:52.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:27:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:27:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:27:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:27:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:27:52.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:27:52.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:27:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:52.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:52.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:52.562 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:52.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:52.619 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:52.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:52.701 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:52.702 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:52.702 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:52.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:52.756 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:52.754+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:52.820 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:52.819+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:52.837 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:52.837 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:52.837 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:52.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:52.930 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:52.929+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:53.013 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:53.013 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:53.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:53.014 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:53.014 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:53.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:53.023 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:53.021+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:53.124 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:53.123+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:53.190 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:53.190 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:53.191 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:53.191 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:53.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:53.225 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:53.223+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:53.326 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:53.325+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:53.367 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:53.367 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:53.367 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:53.367 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:53.368 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:53.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:53.462 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:53.461+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:53.531 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:53.529+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:53.584 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:53.585 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:53.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:53.585 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:53.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.586 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:27:53.586 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:53.586 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.42909966799999, 2023-10-30T06:27:53.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:27:53.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:53.587 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:53.587 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:27:53.587 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:53.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:27:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:53.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:27:53.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:27:53.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:27:53.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:27:53.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:27:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:27:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:27:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:27:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:27:53.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:27:53.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:27:53.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:27:53.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.598 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 218.51592580499999, 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:27:53.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.600 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:53.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:53.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:27:53.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:27:53.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:27:53.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:27:53.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:27:53.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:27:53.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:27:53.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:27:53.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:27:53.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:27:53.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:27:53.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:27:53.610 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.610 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.611 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.611 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:27:53.611 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.85574783600001, 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:53.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:53.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:27:53.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:27:53.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:27:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:27:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:27:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:27:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:27:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:27:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:27:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:27:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:27:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.78032462900001, 2023-10-30T06:27:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:27:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:27:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:27:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:27:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:27:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:27:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:27:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:27:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:27:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:27:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:27:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:27:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:27:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:27:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:27:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:27:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.50920149800001, 2023-10-30T06:27:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:27:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:27:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:27:53.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:27:53.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:27:53.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:27:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:27:53.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:27:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:27:53.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:27:53.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:27:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:27:53.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:27:53.646 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.47892178800001, 2023-10-30T06:27:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:27:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.647 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:27:53.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:27:53.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:27:53.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:27:53.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:27:53.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:27:53.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:27:53.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:27:53.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:27:53.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:27:53.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:27:53.657 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:53.658 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.10501070399999, 2023-10-30T06:27:53.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:27:53.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.658 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:53.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:53.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:27:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:27:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:27:53.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:27:53.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:27:53.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:27:53.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:27:53.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:27:53.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:27:53.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:27:53.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.668 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.669 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:27:53.669 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:53.669 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 217.10051200300001, 2023-10-30T06:27:53.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:53.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:27:53.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:27:53.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:27:53.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:27:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:27:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:27:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:27:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:27:53.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:27:53.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:27:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:27:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:27:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:27:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:27:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:27:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:53.681 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.80890806400001, 2023-10-30T06:27:53.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:27:53.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.681 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:53.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:27:53.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:27:53.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:27:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:27:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:27:53.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:27:53.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:27:53.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:27:53.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:27:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:27:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:27:53.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:27:53.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.691 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.692 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:27:53.692 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:53.692 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.80297074699999, 2023-10-30T06:27:53.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:27:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:53.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:27:53.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:27:53.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:27:53.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:27:53.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:27:53.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:27:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:27:53.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:27:53.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:27:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:27:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.702 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.703 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:27:53.703 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:53.704 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.80167173199999, 2023-10-30T06:27:53.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:27:53.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.704 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:53.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:53.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:27:53.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:27:53.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:27:53.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:27:53.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:27:53.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:27:53.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:27:53.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:27:53.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:27:53.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:27:53.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.714 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.715 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:27:53.715 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:53.715 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.698682924, 2023-10-30T06:27:53.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:27:53.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.716 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:53.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:27:53.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:27:53.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:27:53.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:27:53.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:27:53.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:27:53.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:27:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:27:53.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.726 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:27:53.726 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:53.726 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.426779657, 2023-10-30T06:27:53.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:27:53.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.727 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:27:53.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:27:53.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:27:53.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:27:53.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:27:53.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:27:53.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:27:53.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:27:53.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:27:53.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:27:53.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.736 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.737 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:27:53.737 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.41928146500001, 2023-10-30T06:27:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:27:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:27:53.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:27:53.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.742 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.742 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:27:53.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:27:53.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:27:53.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:27:53.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:27:53.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:27:53.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:27:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:27:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:27:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:27:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.41862026000001, 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:27:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:27:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:27:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:27:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:27:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:27:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:27:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:27:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:27:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:27:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:27:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:27:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:27:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:27:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.406034422, 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:27:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:27:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:27:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:27:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:27:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:27:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:27:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:27:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:27:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:27:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.39994220899999, 2023-10-30T06:27:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:27:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:27:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:27:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:27:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:27:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:27:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:27:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:27:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:27:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:27:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:27:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:27:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:27:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:27:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 216.398946868, 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:27:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:27:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:27:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:27:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:27:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:27:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:27:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:27:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:27:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:27:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:27:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 209.076274297, 2023-10-30T06:27:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:27:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:27:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:27:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:27:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:27:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:27:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:27:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:27:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:27:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:27:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:27:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:27:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 209.042832812, 2023-10-30T06:27:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:27:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:27:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:27:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:27:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:27:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:27:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:27:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:27:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:27:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:27:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:27:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:53.819 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:53.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:27:53.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:53.822 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:53.820+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:53.922 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:53.921+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:53.990 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:53.991 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:53.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:54.042 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:54.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:54.124 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:54.123+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:54.126 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:54.126 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:54.126 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:54.126 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:54.126 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:54.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:54.225 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:54.224+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:54.302 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:54.302 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:54.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:54.302 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:54.303 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:54.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:54.326 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:54.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:54.427 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:54.425+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:54.439 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:54.439 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:54.440 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:54.440 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:54.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:54.527 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:54.526+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:54.574 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:54.574 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:54.574 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:54.575 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:54.575 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:54.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:54.628 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:54.626+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:54.711 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:54.711 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:54.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:54.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:54.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.712 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:27:54.712 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:54.713 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 137.73885029100001, 2023-10-30T06:27:54.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:27:54.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:54.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:54.713 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:54.714 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:27:54.714 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:54.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:27:54.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.714 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:54.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:54.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:54.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:54.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:54.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:27:54.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:27:54.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:54.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:27:54.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:27:54.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:54.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:27:54.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:27:54.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:54.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:27:54.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:27:54.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:54.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:27:54.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:27:54.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:54.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:54.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:54.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:27:54.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:27:54.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:54.724 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:54.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:54.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:54.725 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:54.725 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:54.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:54.729 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:54.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:54.830 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:54.830+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:54.854 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:54.855 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:54.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:54.931 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:54.930+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:55.006 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:55.007 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:55.007 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:55.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:55.056 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:55.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:55.132 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:55.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:55.188 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:55.188 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:55.189 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:55.189 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:55.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:55.233 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:55.233+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:55.350 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:55.349+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:55.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:55.366 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:55.366 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:55.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:27:55.458 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:55.456+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:55.536 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:55.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:55.540 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:55.540 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:55.541 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:55.541 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:55.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:27:55.638 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:55.637+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:55.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-30T06:27:55.739 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:55.737+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:55.769 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:27:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 222.43266753, 2023-10-30T06:27:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:27:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:55.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:27:55.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:27:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:27:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:27:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:27:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:27:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:27:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:27:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:27:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:27:55.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:27:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:27:55.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:27:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:27:55.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:27:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:27:55.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:27:55.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:27:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:27:55.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:27:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:27:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.787 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.111512339, 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.789 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:27:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:27:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:27:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:27:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:27:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:27:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:27:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:27:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:27:55.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:27:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:27:55.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:27:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:27:55.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:27:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:27:55.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:27:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:27:55.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:27:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:27:55.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.803 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.10726778200001, 2023-10-30T06:27:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:27:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:27:55.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:55.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:27:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:27:55.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:27:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:27:55.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:27:55.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:27:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:27:55.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:27:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:27:55.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:27:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:27:55.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:27:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:27:55.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:27:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:27:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:27:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:27:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.818 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:55.819 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.021809357, 2023-10-30T06:27:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:27:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.820 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:27:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:27:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:27:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:27:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:27:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:27:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:27:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:27:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:27:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:27:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:27:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:27:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:27:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:27:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:27:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:27:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:27:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:27:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:27:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:27:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.75256614099999, 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:27:55.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.836 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:27:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:27:55.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:27:55.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:27:55.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:27:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:27:55.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:27:55.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:27:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:27:55.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:27:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:27:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:27:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:27:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:27:55.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:27:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:27:55.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:27:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:27:55.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:27:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:27:55.849 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.850 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.75173473000001, 2023-10-30T06:27:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:27:55.851 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.852 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:27:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:27:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:27:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:27:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:27:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:27:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:27:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:27:55.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:27:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:27:55.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:27:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:27:55.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:27:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:27:55.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:27:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:27:55.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:27:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:27:55.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:27:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:27:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:27:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:27:55.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:27:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:27:55.866 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.867 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.751067603, 2023-10-30T06:27:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:27:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.868 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:27:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:27:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:27:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:27:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:27:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:27:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:27:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:27:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:27:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:27:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:27:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:27:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:27:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:27:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:27:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:27:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:27:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:27:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:27:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.728793532, 2023-10-30T06:27:55.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:27:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.884 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:27:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:27:55.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:27:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:27:55.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:27:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:27:55.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:27:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:27:55.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:27:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:27:55.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:27:55.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:27:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:27:55.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:27:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:27:55.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:27:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:27:55.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:27:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:27:55.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.728145676, 2023-10-30T06:27:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:27:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:27:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:27:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:27:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:27:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:27:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:27:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:27:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:27:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:27:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:27:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:27:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:27:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:27:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:27:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:27:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:27:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:27:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:27:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 219.727459603, 2023-10-30T06:27:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:27:55.914 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.915 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:27:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:27:55.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:27:55.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:27:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:27:55.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:27:55.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:27:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:27:55.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:27:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:27:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:27:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:27:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:27:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:27:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:27:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:27:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:27:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:27:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 218.65657763499999, 2023-10-30T06:27:55.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:27:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:27:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:27:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:27:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:27:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:27:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:27:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:27:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:27:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:27:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:27:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:27:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:27:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:27:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:27:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:27:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:27:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:27:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 218.655913669, 2023-10-30T06:27:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:27:55.945 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.946 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:55.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:27:55.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:27:55.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:27:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:27:55.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:27:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:27:55.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:27:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:27:55.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:27:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:27:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:27:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:27:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:27:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:27:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:27:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:27:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:27:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:27:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 218.62745985999999, 2023-10-30T06:27:55.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:27:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:27:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:27:55.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:27:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:27:55.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:27:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:27:55.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:27:55.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:27:55.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:27:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:27:55.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:27:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:27:55.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:27:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:27:55.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:27:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:27:55.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:55.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.28569336699999, 2023-10-30T06:27:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:27:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:27:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:27:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:27:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:27:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:27:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:27:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:27:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:27:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:27:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:27:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:27:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:27:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:27:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:27:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:27:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:27:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:27:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:27:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:27:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.28494469699999, 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:27:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:27:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:27:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:27:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:27:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:27:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:27:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:27:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:27:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:27:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:27:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:27:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:27:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:27:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:27:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:27:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.28427631400001, 2023-10-30T06:27:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:27:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:27:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:27:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:27:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:27:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:27:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:27:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:27:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:27:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:27:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:27:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:27:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:27:56.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:27:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:27:56.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:27:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:27:56.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.021 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.27442564, 2023-10-30T06:27:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:27:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:56.022 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:27:56.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:27:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:27:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:27:56.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:27:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:27:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:27:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:27:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:27:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:27:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:27:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:27:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:27:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:27:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:27:56.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:27:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.27375182500001, 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:56.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:27:56.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:56.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:27:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:27:56.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:27:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:27:56.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:27:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:27:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:27:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:27:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:27:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:27:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:27:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:27:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:27:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:27:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:27:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:27:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:56.052 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.27310344200001, 2023-10-30T06:27:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:27:56.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:56.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:27:56.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:27:56.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:27:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:27:56.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:27:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:27:56.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:27:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:27:56.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:27:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:27:56.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:27:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:27:56.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:27:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:27:56.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:27:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:27:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:27:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:27:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:27:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:27:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.067 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:56.068 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 211.26898697600001, 2023-10-30T06:27:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:27:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:56.069 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:27:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:56.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:27:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:27:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:56.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:27:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:27:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:56.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:27:56.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:27:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:27:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:27:56.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:27:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:27:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:27:56.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:27:56.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:27:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:27:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:27:56.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:27:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:27:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:27:56.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:27:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:27:56.082 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:56.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:56.083 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:56.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:27:56.086 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:56.084+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:56.187 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:56.186+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:56.258 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:56.258 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:56.259 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:56.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:27:56.287 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:56.287+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:56.388 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:56.387+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:56.392 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:56.392 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:56.393 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:56.393 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:56.393 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:56.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:27:56.442 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:27:56.504 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-30T06:27:56.505 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T06:27:56.505 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T06:27:56.506 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:27:56.508 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:56.507+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:56.520 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:27:56.508+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 106782) UID: 0 2023-10-30T06:27:56.521 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:27:56.509+0000 9f64640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:27:56.608 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:56.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:56.677 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:56.678 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:56.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:56.678 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:56.678 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:56.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:27:56.710 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:56.709+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:56.811 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:56.809+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:56.876 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:56.877 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:56.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:27:56.927 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:56.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:57.031 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:57.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:57.096 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:57.097 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:57.097 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:57.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:27:57.114 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:57.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:57.215 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:57.213+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:57.260 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:42.224 106787== 2023-10-30T06:27:57.260 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:42.224 106787== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:27:57.273 INFO:tasks.ceph.mon.c.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:27:57.282 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:57.282 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:57.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:57.282 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:27:57.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.283 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:27:57.283 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:57.283 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 140.990091541, 2023-10-30T06:27:57.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:27:57.284 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:27:57.284 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:27:57.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:27:57.284 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:27:57.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:57.285 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:27:57.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.285 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:27:57.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:27:57.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:57.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:57.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:27:57.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:27:57.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:27:57.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:27:57.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:27:57.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:27:57.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:27:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:27:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:27:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:27:57.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:27:57.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:27:57.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:27:57.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:27:57.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:27:57.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:27:57.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:27:57.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:27:57.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:27:57.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:27:57.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:27:57.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:27:57.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:27:57.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:57.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:57.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:57.294 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:27:57.294 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:27:57.294 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:57.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:27:57.342 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:57.341+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:57.416 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:57.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:57.425 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:57.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:27:57.517 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:57.516+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:57.599 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:57.600 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:57.600 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:57.600 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:57.600 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:57.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:27:57.649 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:57.648+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:57.718 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:57.717+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:57.774 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:57.775 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:57.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:57.775 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:57.775 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:57.784 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:27:57.819 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:57.817+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:57.940 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:57.938+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:57.953 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:57.954 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:57.954 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:57.954 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:57.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:27:58.021 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:58.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:58.122 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:58.120+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:58.129 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:58.130 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:58.130 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:58.130 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:58.130 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:58.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:27:58.223 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:58.221+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:58.306 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:27:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.150716327, 2023-10-30T06:27:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:27:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:27:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:27:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:27:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:27:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:27:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:27:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:27:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:27:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:27:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:27:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:27:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:27:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:27:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:27:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:27:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:27:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:27:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:27:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 223.237542464, 2023-10-30T06:27:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:27:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:27:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:27:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:27:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:27:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:27:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:27:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:27:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:27:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:27:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:27:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:27:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:27:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:27:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:27:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.57736449500001, 2023-10-30T06:27:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:27:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:27:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:27:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:27:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:27:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:27:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:27:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:27:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:27:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:27:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:27:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:27:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:27:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:27:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:27:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:27:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.50194128800001, 2023-10-30T06:27:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:27:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:27:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:27:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:27:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:27:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:27:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:27:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:27:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:27:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:27:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:27:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:27:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:27:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:27:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.23081815699999, 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:27:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:27:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:27:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:27:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:27:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:27:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:27:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:27:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:27:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:27:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:27:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:27:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:27:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:27:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:27:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:27:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.20053844700001, 2023-10-30T06:27:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:27:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:27:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:27:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:27:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:27:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:27:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:27:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:27:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:27:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:27:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:27:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:27:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:27:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:27:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:27:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:27:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:27:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.826627363, 2023-10-30T06:27:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:27:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:27:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:27:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:27:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:27:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:27:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:27:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:27:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:27:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:27:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:27:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:27:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:27:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:27:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:27:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:27:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:27:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.82212866200001, 2023-10-30T06:27:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:27:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:27:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:27:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:27:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:27:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:27:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:27:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:27:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:27:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:27:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:27:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:27:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:27:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:27:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.53052472300001, 2023-10-30T06:27:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:27:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:27:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:27:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:27:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:27:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:27:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:27:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:27:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:27:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:27:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:27:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:27:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:27:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:27:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:27:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.52458740599999, 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:27:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:27:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:27:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:27:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:27:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:27:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:27:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:27:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:27:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:27:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:27:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:27:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:27:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:27:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.52328839099999, 2023-10-30T06:27:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:27:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:27:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:27:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:27:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:27:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:27:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:27:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:27:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:27:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:27:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:27:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:27:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:27:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:27:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:27:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:27:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:27:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:27:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.420299583, 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:27:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:27:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:27:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:27:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:27:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:27:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:27:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:27:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:27:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:27:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:27:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:27:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:27:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:27:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:27:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:27:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:27:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.148396316, 2023-10-30T06:27:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:27:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:27:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:27:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:27:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:27:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:27:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:27:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:27:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:27:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:27:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:27:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:27:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:27:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:27:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:27:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:27:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:27:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.14089812399999, 2023-10-30T06:27:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:27:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:27:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:27:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:27:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:27:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:27:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:27:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:27:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:27:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:27:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:27:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:27:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:27:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:27:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:27:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:58.476 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.14023691899999, 2023-10-30T06:27:58.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:27:58.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.476 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:27:58.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:27:58.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:27:58.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:27:58.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:27:58.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:27:58.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:27:58.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:27:58.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:27:58.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:27:58.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:27:58.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:27:58.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:27:58.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.486 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.487 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:27:58.487 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:58.487 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.12765108100001, 2023-10-30T06:27:58.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:27:58.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.488 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:27:58.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:27:58.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:27:58.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:27:58.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:27:58.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:27:58.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:27:58.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:27:58.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:27:58.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:27:58.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:27:58.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:27:58.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:27:58.497 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.498 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.498 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.498 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.12155886799999, 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.499 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:58.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:27:58.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:27:58.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:27:58.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:27:58.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:27:58.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:27:58.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:27:58.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:27:58.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:27:58.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:27:58.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:27:58.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:27:58.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:27:58.509 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.509 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.509 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.510 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:27:58.510 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:58.510 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 221.120563527, 2023-10-30T06:27:58.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:27:58.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.511 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:27:58.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:27:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:27:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:27:58.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:27:58.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:27:58.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:27:58.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:27:58.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:27:58.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:27:58.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:27:58.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:27:58.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:27:58.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:27:58.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:27:58.521 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.521 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.521 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:27:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 213.797890956, 2023-10-30T06:27:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:58.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:27:58.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:27:58.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:27:58.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:27:58.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:27:58.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:27:58.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:27:58.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:27:58.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:27:58.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:27:58.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:27:58.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:27:58.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:27:58.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:27:58.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:27:58.533 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.533 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.533 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.534 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:27:58.534 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:58.534 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 213.76444947100001, 2023-10-30T06:27:58.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:27:58.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:58.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:27:58.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:58.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:58.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:27:58.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:27:58.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:58.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:27:58.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:27:58.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:58.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:27:58.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:27:58.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:58.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:27:58.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:27:58.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:27:58.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:27:58.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:27:58.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:58.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:58.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:58.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:27:58.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:27:58.544 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.544 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.545 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.545 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:58.545 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:58.545 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:58.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:27:58.547 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:27:58.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:27:58.647 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:58.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:58.714 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:58.715 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:58.717 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:58.717 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:58.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:27:58.765 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:58.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:58.849 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:27:58.847+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:27:58.890 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:58.890 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:58.890 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:58.891 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:58.891 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:58.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:27:58.950 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:27:58.948+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:27:59.025 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:59.026 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:59.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:59.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:27:59.026 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:59.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:27:59.078 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:27:59.077+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:27:59.152 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:59.151+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:59.160 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:59.160 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:59.160 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:27:59.160 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:59.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:27:59.253 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:59.252+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:59.335 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:59.335 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:27:59.335 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:27:59.336 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:27:59.336 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:59.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:27:59.389 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:59.387+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:59.458 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:27:59.456+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:27:59.473 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:27:59.473 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:27:59.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:27:59.474 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:27:59.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.474 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:27:59.474 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:59.475 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 142.501199407, 2023-10-30T06:27:59.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:27:59.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:27:59.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:27:59.475 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:27:59.476 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:27:59.476 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:27:59.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:27:59.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.476 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:27:59.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:27:59.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:59.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:59.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:27:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:27:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:27:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:27:59.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:27:59.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:27:59.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:27:59.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:27:59.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:27:59.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:27:59.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:27:59.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:27:59.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:27:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:27:59.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:27:59.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:27:59.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:27:59.484 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:59.484 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:59.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:59.485 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:27:59.485 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:27:59.485 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:27:59.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:27:59.557 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:59.555+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:59.623 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:27:59.624 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:59.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:27:59.658 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:27:59.656+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:27:59.758 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:59.757+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:27:59.798 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:59.798 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:27:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:27:59.799 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:27:59.799 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:59.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:27:59.859 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:27:59.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:27:59.934 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:27:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:27:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:27:59.935 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:27:59.935 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:27:59.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:27:59.959 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:27:59.958+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:00.060 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:00.059+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:00.113 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:00.114 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:00.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:00.161 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:00.160+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:00.250 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:00.251 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:00.251 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:00.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:00.262 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:00.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:00.363 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:00.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:00.445 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:00.445 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:00.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 227.10485078900001, 2023-10-30T06:28:00.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:00.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:00.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:00.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:00.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:00.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:00.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:00.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:00.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:00.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:00.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:00.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:00.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:00.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:00.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:00.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:00.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:00.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:00.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:00.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:00.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:00.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:00.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:00.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:00.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:00.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:00.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:00.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:00.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:00.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.466 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.466 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.467 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.467 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:00.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.78369559800001, 2023-10-30T06:28:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:00.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:00.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:00.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:00.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:00.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:00.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:00.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:00.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:00.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:00.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:00.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:00.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:00.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.77945104099999, 2023-10-30T06:28:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.693992616, 2023-10-30T06:28:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.4247494, 2023-10-30T06:28:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.42391798899999, 2023-10-30T06:28:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.423250862, 2023-10-30T06:28:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.40097679100001, 2023-10-30T06:28:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.400328935, 2023-10-30T06:28:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.39964286200001, 2023-10-30T06:28:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 223.328760894, 2023-10-30T06:28:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 223.32809692800001, 2023-10-30T06:28:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 223.299643119, 2023-10-30T06:28:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.957876626, 2023-10-30T06:28:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.95712795599999, 2023-10-30T06:28:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.95645957299999, 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.94660889900001, 2023-10-30T06:28:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.94593508400001, 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.94528670099999, 2023-10-30T06:28:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 215.94117023499999, 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:00.786 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:00.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:00.788 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:00.786+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:00.902 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:00.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:00.916 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:00.916 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:00.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:01.011 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:01.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:01.090 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:01.088+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:01.092 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:01.092 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:01.092 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:01.093 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:01.093 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:01.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:01.191 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:01.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:01.276 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:01.276 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:01.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:01.277 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:01.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:01.330 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:01.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:01.392 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:01.390+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:01.412 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:01.412 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:01.413 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:01.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:01.493 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:01.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:01.589 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:01.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:01.589 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:01.590 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:01.590 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:01.595 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:01.592+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:01.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:01.695 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:01.693+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:01.772 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:01.772 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:01.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:01.773 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:01.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.773 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:01.773 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:01.774 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 145.48044943100001, 2023-10-30T06:28:01.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:01.774 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:01.774 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:01.774 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:01.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:01.775 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:01.775 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:01.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.775 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:01.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:01.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:01.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:01.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:01.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:01.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:01.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:01.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:01.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:01.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:01.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:01.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:01.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:01.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:01.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:01.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:01.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:01.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:01.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:01.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:01.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:01.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:01.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:01.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:01.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:01.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:01.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:01.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:01.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:01.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:01.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:01.785 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:01.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:01.828 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:01.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:01.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-30T06:28:01.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-30T06:28:01.897 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:01.896+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:01.926 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:01.926 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:01.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:02.020 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:02.019+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:02.099 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:02.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:02.102 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:02.102 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:02.102 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:02.103 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:02.103 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:02.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:02.199 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:02.198+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:02.278 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:02.279 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:02.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:02.279 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:02.279 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:02.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:02.331 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:02.329+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:02.402 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:02.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:02.457 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:02.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:02.509 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:02.506+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:02.604 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:02.603+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:02.608 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:28:02.608 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-30T06:28:02.609 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T06:28:02.609 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T06:28:02.609 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:28:02.622 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:28:02.612+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 108041) UID: 0 2023-10-30T06:28:02.624 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:28:02.613+0000 9f64640 -1 mon.a@0(probing) e1 *** Got Signal Terminated *** 2023-10-30T06:28:02.631 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:02.631 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:02.632 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:02.632 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:02.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:02.705 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:02.704+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:02.806 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:02.805+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:02.849 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.694235905, 2023-10-30T06:28:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 227.781062042, 2023-10-30T06:28:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 227.12088407300001, 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 227.04546086600001, 2023-10-30T06:28:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.77433773499999, 2023-10-30T06:28:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.74405802499999, 2023-10-30T06:28:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.370146941, 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.36564824000001, 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.07404430099999, 2023-10-30T06:28:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:02.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:02.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:02.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:02.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:02.954 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:02.955 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.068106984, 2023-10-30T06:28:02.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:02.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.955 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:02.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:02.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:02.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:02.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:02.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:02.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:02.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:02.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:02.965 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.965 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.965 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.966 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:02.966 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:02.966 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 226.066807969, 2023-10-30T06:28:02.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:02.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:02.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:02.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:02.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:02.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:02.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:02.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:02.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:02.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:02.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:02.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:02.976 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.976 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.976 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.977 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:02.977 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:02.977 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.963819161, 2023-10-30T06:28:02.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:02.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.978 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:02.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:02.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:02.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.982 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:02.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:02.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:02.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:02.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:02.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:02.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:02.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:02.988 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.988 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.988 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.691915894, 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:02.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:02.990 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:02.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:02.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:02.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:02.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:02.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:02.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:02.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:02.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:02.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:02.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:02.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:02.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:02.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:02.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:02.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:02.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:02.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:02.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:02.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.000 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:03.000 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:03.000 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.68441770199999, 2023-10-30T06:28:03.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:03.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:03.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:03.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:03.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:03.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:03.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:03.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:03.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:03.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:03.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:03.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:03.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:03.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.011 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.012 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:03.012 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:03.012 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.68375649699999, 2023-10-30T06:28:03.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:03.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.013 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:03.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:03.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:03.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:03.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:03.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:03.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:03.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:03.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:03.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:03.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:03.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:03.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:03.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:03.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.025 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:03.026 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:03.026 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.67117065900001, 2023-10-30T06:28:03.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:03.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.027 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:03.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:03.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:03.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:03.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:03.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:03.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:03.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:03.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:03.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:03.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:03.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:03.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:03.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:03.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:03.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:03.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:03.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:03.039 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.040 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.041 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:03.041 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:03.041 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.665078446, 2023-10-30T06:28:03.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:03.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.042 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:03.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:03.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:03.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:03.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:03.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:03.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:03.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:03.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:03.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:03.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:03.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:03.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:03.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:03.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:03.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:03.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:03.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:03.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.053 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.054 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:03.054 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:03.054 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 225.664083105, 2023-10-30T06:28:03.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:03.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.055 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:03.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:03.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:03.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:03.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:03.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:03.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:03.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:03.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:03.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:03.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:03.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:03.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:03.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:03.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:03.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:03.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:03.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.066 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.066 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.067 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:03.067 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:03.067 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 218.341410534, 2023-10-30T06:28:03.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:03.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.068 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:03.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:03.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:03.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:03.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:03.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:03.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:03.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 218.30796904900001, 2023-10-30T06:28:03.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:03.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:03.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:03.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:03.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:03.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:03.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:03.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:03.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:03.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:03.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:03.095 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:03.098 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:03.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:03.163 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:28.718 108046== 2023-10-30T06:28:03.163 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:28.718 108046== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:28:03.172 INFO:tasks.ceph.mon.a.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:28:03.222 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:03.219+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:03.224 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:03.224 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:03.224 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:03.225 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.233 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:03.299 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:03.298+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:03.364 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:03.365 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:03.365 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:03.365 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:03.365 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:03.419 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:03.418+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:03.500 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:03.499+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:03.505 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:03.505 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:03.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:03.506 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:03.506 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:03.601 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:03.600+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:03.693 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:03.694 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:03.694 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:03.694 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.702 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:03.700+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:03.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:03.804 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:03.802+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:03.871 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:03.871 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:03.871 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:03.872 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:03.872 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:03.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:03.925 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:03.923+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.005 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:04.004+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:04.014 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:04.014 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:04.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:04.015 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:04.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.015 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:04.015 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:04.015 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 147.04155122700001, 2023-10-30T06:28:04.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:04.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:04.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:04.016 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:04.016 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:04.017 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:04.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:04.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.017 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:04.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:04.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:04.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:04.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:04.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:04.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:04.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:04.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:04.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:04.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:04.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:04.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:04.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:04.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:04.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:04.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:04.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:04.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:04.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:04.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:04.025 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:04.026 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:04.026 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:04.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:04.106 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:04.105+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.158 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:04.159 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:04.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:04.207 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:04.206+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.297 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.297 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:04.297 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:04.298 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:04.298 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:04.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:04.308 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:04.306+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.409 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:04.408+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:04.433 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:04.434 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:04.435 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:04.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:04.510 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:04.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.570 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:04.570 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:04.570 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:04.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:04.611 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:04.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:04.705 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.705 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:04.706 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:04.706 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:04.712 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:04.710+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:04.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:04.813 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:04.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:04.915 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:04.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:04.936 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 231.59948597600001, 2023-10-30T06:28:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.27833078500001, 2023-10-30T06:28:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.27408622799999, 2023-10-30T06:28:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.188627803, 2023-10-30T06:28:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.919384587, 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.91855317599999, 2023-10-30T06:28:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.917886049, 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.89561197800001, 2023-10-30T06:28:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.894964122, 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 228.89427804900001, 2023-10-30T06:28:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 227.823396081, 2023-10-30T06:28:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 227.82273211500001, 2023-10-30T06:28:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 227.794278306, 2023-10-30T06:28:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.452511813, 2023-10-30T06:28:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.45176314299999, 2023-10-30T06:28:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.45109475999999, 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.44124408600001, 2023-10-30T06:28:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:05.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.44057027100001, 2023-10-30T06:28:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:05.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:05.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:05.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:05.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:05.215 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.216 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.43992188799999, 2023-10-30T06:28:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.217 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.218 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:05.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:05.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:05.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:05.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.230 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:05.231 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 220.43580542199999, 2023-10-30T06:28:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:05.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:05.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:05.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:05.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:05.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:05.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:05.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:05.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:05.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:05.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:05.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:05.246 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:05.246 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:05.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:05.248 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:05.246+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:05.348 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:05.347+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:05.416 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:05.416 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:05.416 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:05.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:05.465 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:05.464+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:05.551 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:05.549+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:05.591 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:05.592 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:05.592 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:05.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:05.687 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:05.685+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:05.753 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:05.751+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:05.770 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:05.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:05.771 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:05.771 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:05.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:05.864 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:05.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:05.947 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:05.948 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:05.948 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:05.948 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:05.954 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:05.952+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:05.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:06.055 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:06.053+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:06.121 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:06.122 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:06.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:06.176 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:06.174+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:06.256 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:06.255+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:06.260 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:06.261 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:06.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:06.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:06.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.261 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:06.262 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:06.262 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 149.969016788, 2023-10-30T06:28:06.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:06.262 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:06.262 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:06.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:06.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:06.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:06.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:06.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:06.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:06.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:06.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:06.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:06.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:06.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:06.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:06.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:06.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:06.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:06.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:06.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:06.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:06.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:06.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:06.272 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:06.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:06.358 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:06.357+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:06.411 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:06.412 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:06.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:06.467 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:06.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:06.548 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:06.549 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:06.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:06.598 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:06.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:06.660 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:06.659+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:06.723 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:06.724 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:06.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:06.761 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:06.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:06.862 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:06.860+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:06.902 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:06.902 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:06.902 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:06.903 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:06.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:06.966 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:06.965+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:07.041 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:07.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:07.042 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:07.042 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:07.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:07.064 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:07.062+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:07.165 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:07.163+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:07.257 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.102339637, 2023-10-30T06:28:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 232.189165774, 2023-10-30T06:28:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.52898780500001, 2023-10-30T06:28:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.45356459800001, 2023-10-30T06:28:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.18244146699999, 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:07.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:07.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:07.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:07.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:07.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:07.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:07.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:07.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:07.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:07.317 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.317 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.318 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.318 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:07.318 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:07.319 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.15216175699999, 2023-10-30T06:28:07.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:07.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.319 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:07.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:07.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:07.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:07.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:07.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:07.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:07.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:07.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:07.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:07.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:07.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:07.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:07.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:07.329 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.329 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.330 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.330 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:07.330 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.778250673, 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:07.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:07.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:07.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:07.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:07.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:07.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:07.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:07.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:07.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:07.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:07.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:07.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:07.341 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.341 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.341 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.342 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:07.342 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:07.342 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.77375197200001, 2023-10-30T06:28:07.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:07.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.343 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:07.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:07.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:07.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:07.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:07.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:07.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:07.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:07.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:07.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.354 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:07.354 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:07.354 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.48214803299999, 2023-10-30T06:28:07.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:07.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.355 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:07.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:07.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:07.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:07.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:07.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:07.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:07.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:07.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:07.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:07.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:07.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:07.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.364 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:07.365 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:07.366 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.476210716, 2023-10-30T06:28:07.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:07.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.366 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:07.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:07.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:07.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:07.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:07.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:07.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:07.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:07.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:07.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.376 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.377 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:07.377 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:07.377 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.474911701, 2023-10-30T06:28:07.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:07.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.378 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:07.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:07.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:07.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:07.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:07.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:07.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:07.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:07.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:07.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:07.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:07.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:07.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.388 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.371922893, 2023-10-30T06:28:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.390 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:07.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:07.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:07.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:07.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:07.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:07.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:07.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:07.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:07.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:07.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:07.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:07.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:07.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:07.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:07.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.400 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.10001962600001, 2023-10-30T06:28:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.402 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:07.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:07.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:07.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:07.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:07.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:07.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:07.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:07.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:07.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:07.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:07.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:07.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:07.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:07.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.413 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:07.413 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:07.413 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.09252143399999, 2023-10-30T06:28:07.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:07.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.414 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:07.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:07.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:07.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:07.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:07.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:07.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:07.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:07.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:07.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:07.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:07.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:07.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:07.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:07.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:07.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:07.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.424 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.424 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.425 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:07.425 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:07.425 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.09186022899999, 2023-10-30T06:28:07.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:07.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.426 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:07.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:07.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:07.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:07.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:07.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:07.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:07.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:07.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:07.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:07.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:07.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:07.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:07.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.436 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.07927439100001, 2023-10-30T06:28:07.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:07.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.438 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:07.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:07.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:07.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:07.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:07.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:07.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:07.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:07.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:07.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:07.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:07.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:07.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.449 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:07.449 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:07.449 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.073182178, 2023-10-30T06:28:07.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:07.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.450 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:07.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:07.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:07.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:07.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:07.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:07.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:07.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:07.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:07.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:07.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:07.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:07.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:07.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:07.459 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.460 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.460 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 230.072186837, 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.461 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:07.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:07.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:07.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:07.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:07.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:07.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:07.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:07.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:07.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:07.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:07.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.74951426600001, 2023-10-30T06:28:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:07.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.473 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:07.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:07.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:07.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:07.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:07.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:07.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:07.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:07.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:07.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:07.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:07.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:07.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:07.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:07.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:07.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.483 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.483 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.484 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:07.484 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:07.484 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 222.71607278100001, 2023-10-30T06:28:07.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:07.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:07.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:07.485 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:07.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:07.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:07.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:07.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:07.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:07.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:07.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:07.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:07.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:07.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:07.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:07.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:07.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:07.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:07.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:07.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:07.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:07.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:07.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:07.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:07.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:07.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:07.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:07.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:07.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:07.495 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:07.496 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:07.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:07.498 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:07.496+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:07.599 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:07.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:07.623 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:07.624 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:07.624 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:07.624 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:07.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:07.699 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:07.698+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:07.760 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:07.760 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:07.760 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:07.760 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:07.761 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:07.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:07.801 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:07.799+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:07.894 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:07.895 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:07.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:07.895 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:07.895 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:07.901 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:07.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:07.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:08.002 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:08.001+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:08.033 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:08.033 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:08.034 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:08.034 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:08.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:08.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-30T06:28:08.103 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-30T06:28:08.104 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:08.102+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:08.171 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:08.171 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:08.171 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:08.172 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:08.172 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:08.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:08.221 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:08.220+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:08.306 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:08.305+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:08.348 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:08.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.350 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:08.350 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:08.350 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 151.37616987800001, 2023-10-30T06:28:08.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:08.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:08.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:08.351 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:08.351 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:08.351 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:08.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:08.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.352 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:08.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:08.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:08.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:08.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:08.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:08.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:08.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:08.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:08.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:08.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:08.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:08.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:08.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:08.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:08.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:08.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:08.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:08.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:08.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:08.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:08.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:08.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:08.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:08.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:08.361 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:08.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:08.361 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:08.361 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:08.362 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:08.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:08.408 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:08.406+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:08.501 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:08.502 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:08.502 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:08.509 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:08.508+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:08.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:08.610 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:08.608+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:08.636 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:08.636 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:08.637 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:08.637 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:08.637 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:08.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:08.710 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:28:08.711 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-10-30T06:28:08.711 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T06:28:08.712 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2023-10-30T06:28:08.712 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T06:28:08.712 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-30T06:28:08.713 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:08.709+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:08.715 INFO:tasks.ceph.mon.b:Started 2023-10-30T06:28:08.715 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T06:28:08.716 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-30T06:28:08.718 INFO:tasks.ceph.mon.c:Started 2023-10-30T06:28:08.718 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:28:08.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:08.787 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:08.787 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:08.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:08.788 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:08.788 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:08.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:08.812 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:08.810+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:08.912 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:08.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:08.929 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:08.938 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:09.013 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:09.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:09.067 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:09.068 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:09.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:09.115 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:09.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:09.216 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:09.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:09.305 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:09.305 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 235.967755751, 2023-10-30T06:28:09.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.307 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:09.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:09.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:09.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:09.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:09.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:09.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:09.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:09.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:09.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.323 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.64660056, 2023-10-30T06:28:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.324 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:09.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:09.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:09.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:09.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:09.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:09.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:09.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:09.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:09.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:09.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:09.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.338 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.339 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.642356003, 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.340 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:09.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.55689757799999, 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.28765436200001, 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.286822951, 2023-10-30T06:28:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.28615582399999, 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.26388175299999, 2023-10-30T06:28:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.26323389699999, 2023-10-30T06:28:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 233.26254782399999, 2023-10-30T06:28:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 232.19166585599999, 2023-10-30T06:28:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 232.19100189, 2023-10-30T06:28:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 232.16254808100001, 2023-10-30T06:28:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.82078158799999, 2023-10-30T06:28:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.82003291800001, 2023-10-30T06:28:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.81936453500001, 2023-10-30T06:28:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.809513861, 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.808840046, 2023-10-30T06:28:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.808191663, 2023-10-30T06:28:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 224.804075197, 2023-10-30T06:28:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:09.638 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:09.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:09.641 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:09.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:09.740 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:09.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:09.810 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:09.811 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:09.811 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:09.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:09.865 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:09.863+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:09.943 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:09.941+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:09.955 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:09.955 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:09.956 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:09.956 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:09.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:10.044 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:10.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:10.139 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:10.140 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:10.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:10.140 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:10.140 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:10.144 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:10.142+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:10.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:10.245 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:10.244+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:10.322 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:10.323 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:10.323 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:10.323 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:10.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:10.381 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:10.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:10.447 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:10.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:10.464 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:10.465 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:10.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:10.557 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:10.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:10.646 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:10.646 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:10.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:10.646 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:10.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.647 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:10.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:10.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 154.35406137699999, 2023-10-30T06:28:10.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:10.648 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:10.648 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:10.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:10.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:10.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:10.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:10.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:10.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:10.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:10.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:10.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:10.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:10.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:10.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:10.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:10.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:10.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:10.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:10.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:10.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:10.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:10.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:10.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:10.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:10.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:10.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:10.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:10.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:10.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:10.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:10.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:10.659 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:10.659 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:10.660 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:10.660 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:10.660 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:10.662 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:10.662 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:10.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:10.763 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:10.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:10.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:10.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:10.864 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:10.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:10.932 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:10.932 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:10.933 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:10.933 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:10.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:10.983 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:10.982+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:11.065 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:11.064+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:11.112 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:11.112 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:11.113 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:11.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:11.166 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:11.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:11.277 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:11.276+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:11.292 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:11.293 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:11.293 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:11.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:11.368 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:11.366+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:11.469 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:11.468+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:11.474 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:11.474 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:11.475 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:11.475 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:11.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:11.570 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:11.569+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:11.654 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.49788877, 2023-10-30T06:28:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 236.58471490700001, 2023-10-30T06:28:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.92453693799999, 2023-10-30T06:28:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.84911373099999, 2023-10-30T06:28:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.57799059999999, 2023-10-30T06:28:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.54771088999999, 2023-10-30T06:28:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.17379980600001, 2023-10-30T06:28:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 235.16930110499999, 2023-10-30T06:28:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.87769716599999, 2023-10-30T06:28:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.871759849, 2023-10-30T06:28:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.870460834, 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.76747202600001, 2023-10-30T06:28:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.49556875900001, 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.48807056699999, 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.48740936199999, 2023-10-30T06:28:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.47482352399999, 2023-10-30T06:28:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.468731311, 2023-10-30T06:28:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 234.46773597000001, 2023-10-30T06:28:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:11.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:11.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:11.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:11.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:11.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:11.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.873 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 227.14506339900001, 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.874 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:11.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 227.11162191400001, 2023-10-30T06:28:11.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:11.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:11.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:11.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:11.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:11.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:11.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:11.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:11.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:11.896 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:11.897 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:11.898 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:11.899 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:11.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:11.999 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:11.997+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:12.070 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.071 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:12.071 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:12.071 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:12.100 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:12.099+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:12.202 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:12.200+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:12.207 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.207 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:12.208 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:12.208 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:12.208 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:12.302 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:12.301+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:12.385 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.385 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:12.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:12.385 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:12.386 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:12.403 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:12.402+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:12.504 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:12.503+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:12.523 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.524 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:12.524 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:12.524 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:12.606 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:12.604+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:12.661 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.661 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:12.662 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:12.662 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:12.662 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:12.714 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:12.713+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:12.801 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.802 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:12.803 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:12.803 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 155.82896870499999, 2023-10-30T06:28:12.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:12.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:12.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.805 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:12.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:12.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:12.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:12.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:12.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:12.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:12.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:12.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:12.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:12.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:12.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:12.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:12.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:12.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:12.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:12.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:12.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:12.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:12.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:12.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:12.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:12.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:12.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:12.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:12.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:12.813 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:12.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:12.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:12.814 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:12.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:12.816 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:12.814+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:12.949 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:12.947+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:12.963 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:12.964 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:12.964 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:12.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:13.017 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:13.016+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:13.143 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:13.142+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:13.157 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:13.158 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:13.158 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:13.158 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:13.159 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:13.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:13.219 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:13.218+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:13.320 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:13.319+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:13.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:13.365 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:13.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:13.365 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:13.366 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:13.374 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:13.421 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:13.420+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:13.519 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:13.519 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:13.519 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:13.522 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:13.521+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:13.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:13.623 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:13.622+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:13.700 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:13.700 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:13.701 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:13.701 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:13.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:13.752 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:13.749+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:13.826 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:13.824+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:13.927 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:13.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:13.948 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:13.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 240.60913593199999, 2023-10-30T06:28:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:13.950 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:13.951 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:13.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:13.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:13.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:13.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:13.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:13.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:13.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:13.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:13.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:13.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:13.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:13.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:13.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.28798074100001, 2023-10-30T06:28:13.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:13.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:13.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:13.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:13.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:13.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:13.985 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.986 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.28373618399999, 2023-10-30T06:28:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:13.987 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:13.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:13.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:13.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:13.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:13.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:13.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:13.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:14.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:14.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.002 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:14.003 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.19827775900001, 2023-10-30T06:28:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.004 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:14.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:14.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:14.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:14.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:14.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.018 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.019 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.929034543, 2023-10-30T06:28:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.020 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:14.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:14.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:14.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:14.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:14.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:14.036 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.92820313199999, 2023-10-30T06:28:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.037 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.038 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.038 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:14.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.038 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:14.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:14.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:14.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:14.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:14.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:14.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:14.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:14.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:14.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:14.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:14.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:14.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:14.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:14.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:14.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:14.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.052 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.052 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.052 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.92753600500001, 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.053 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.054 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.054 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:14.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.055 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:14.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:14.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:14.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:14.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:14.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:14.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:14.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:14.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:14.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:14.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:14.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:14.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:14.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:14.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:14.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:14.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:14.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:14.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:14.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:14.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.069 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.069 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.070 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.070 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:14.070 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.90526193400001, 2023-10-30T06:28:14.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:14.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:14.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.072 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:14.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:14.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:14.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:14.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:14.085 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.90461407800001, 2023-10-30T06:28:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:14.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:14.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:14.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:14.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:14.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:14.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 237.90392800500001, 2023-10-30T06:28:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.101 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:14.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:14.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:14.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:14.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 236.833046037, 2023-10-30T06:28:14.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:14.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:14.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:14.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:14.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:14.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:14.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:14.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:14.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:14.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:14.131 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 236.83238207100001, 2023-10-30T06:28:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:14.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:14.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:14.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:14.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 236.803928262, 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.147 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:14.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:14.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:14.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:14.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:14.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:14.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:14.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:14.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.161 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:14.162 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.46216176900001, 2023-10-30T06:28:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.163 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:14.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:14.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:14.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:14.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:14.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:14.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.177 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.177 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.178 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:14.178 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.461413099, 2023-10-30T06:28:14.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:14.178 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.178 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:14.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:14.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:14.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:14.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:14.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:14.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:14.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:14.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:14.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:14.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:14.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:14.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:14.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:14.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:14.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:14.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.193 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:14.193 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.46074471599999, 2023-10-30T06:28:14.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:14.194 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.194 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.194 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.195 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:14.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.195 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:14.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:14.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:14.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:14.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:14.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:14.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:14.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:14.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:14.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:14.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:14.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:14.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:14.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:14.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:14.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:14.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:14.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:14.207 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.208 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:14.209 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.45089404199999, 2023-10-30T06:28:14.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:14.209 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.209 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.209 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.210 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.210 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:14.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.210 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:14.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:14.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:14.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:14.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:14.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:14.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:14.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:14.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:14.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:14.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:14.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:14.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:14.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:14.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:14.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:14.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:14.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.224 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.224 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:14.224 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.45022022699999, 2023-10-30T06:28:14.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:14.224 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:14.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.226 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:14.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:14.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:14.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:14.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:14.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:14.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:14.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:14.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:14.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:14.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:14.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:14.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:14.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:14.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.44957184399999, 2023-10-30T06:28:14.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:14.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.240 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:14.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.241 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:14.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:14.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:14.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:14.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:14.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:14.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:14.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:14.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:14.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:14.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:14.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:14.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:14.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:14.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:14.253 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.253 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:14.254 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 229.44545537799999, 2023-10-30T06:28:14.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:14.255 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:14.255 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:14.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:14.255 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:14.256 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:14.256 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:14.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.256 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:14.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:14.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:14.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:14.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:14.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:14.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:14.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:14.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:14.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:14.270 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:14.271 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:14.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:14.273 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:14.271+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:14.374 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:14.372+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:14.442 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:14.442 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:14.443 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:14.443 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:14.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:14.496 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:14.494+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:14.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-30T06:28:14.576 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:14.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:14.586 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:14.586 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:14.587 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:14.587 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:14.587 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:14.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:14.678 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:14.676+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:14.765 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:14.766 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:14.767 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:14.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:14.778 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:14.777+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:14.881 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:14.878+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:14.903 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:14.904 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:14.904 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:14.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:14.980 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:14.979+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:15.042 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:15.043 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:15.043 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:15.043 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:15.043 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:15.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:15.081 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:15.079+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:15.182 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:15.181+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:15.186 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:15.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 158.895093097, 2023-10-30T06:28:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:15.188 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:15.189 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:15.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:15.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:15.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:15.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:15.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:15.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:15.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:15.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:15.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:15.199 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:15.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:15.283 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:15.281+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:15.329 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:15.329 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:15.330 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:15.330 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:15.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:15.424 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:15.422+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:15.484 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:15.483+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:15.507 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:15.507 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:15.507 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:15.507 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:15.508 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:15.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:15.585 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:15.584+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:15.684 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:15.684 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:15.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:15.685 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:15.685 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:15.686 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:15.685+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:15.693 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:15.787 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:15.786+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:15.862 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:15.862 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:15.862 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:15.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:15.888 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:15.886+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:15.991 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:15.989+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:16.041 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:16.042 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:16.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:16.090 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:16.089+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:16.191 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:16.190+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:16.220 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.06333640299999, 2023-10-30T06:28:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 241.15016254, 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 240.48998457100001, 2023-10-30T06:28:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 240.41456136400001, 2023-10-30T06:28:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:16.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:16.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:16.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:16.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:16.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:16.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:16.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:16.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:16.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:16.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:16.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:16.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:16.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:16.268 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.269 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.269 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.269 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 240.14343823300001, 2023-10-30T06:28:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:16.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:16.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:16.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:16.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:16.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 240.11315852300001, 2023-10-30T06:28:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.739247439, 2023-10-30T06:28:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.303 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.304 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.304 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:16.304 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:16.305 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.73474873800001, 2023-10-30T06:28:16.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:16.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.305 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:16.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:16.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:16.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:16.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:16.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:16.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:16.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:16.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:16.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:16.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:16.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:16.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:16.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.316 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:16.316 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:16.316 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.44314479900001, 2023-10-30T06:28:16.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:16.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.317 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:16.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:16.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:16.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:16.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:16.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:16.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:16.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:16.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:16.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:16.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:16.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.330 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:16.330 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:16.330 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.43720748199999, 2023-10-30T06:28:16.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:16.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.331 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:16.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:16.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:16.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:16.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:16.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:16.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:16.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:16.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:16.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:16.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:16.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.340 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.341 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.341 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:16.341 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:16.342 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.43590846699999, 2023-10-30T06:28:16.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:16.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.342 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:16.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:16.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:16.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:16.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:16.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:16.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:16.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:16.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:16.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:16.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.353 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:16.353 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:16.353 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.332919659, 2023-10-30T06:28:16.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:16.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.354 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:16.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:16.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:16.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:16.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:16.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:16.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:16.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:16.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:16.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:16.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:16.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:16.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.363 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.364 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.364 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:16.364 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.061016392, 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:16.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:16.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:16.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:16.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:16.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:16.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:16.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:16.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:16.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:16.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:16.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:16.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:16.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:16.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.375 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.375 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.05351820000001, 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:16.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.377 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:16.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:16.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:16.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:16.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:16.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:16.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:16.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:16.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:16.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:16.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:16.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:16.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:16.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:16.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.387 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.387 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:16.388 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:16.388 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.05285699500001, 2023-10-30T06:28:16.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:16.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:16.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:16.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:16.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:16.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:16.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:16.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:16.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:16.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:16.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:16.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:16.398 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.398 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.398 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.04027115700001, 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:16.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.400 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:16.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:16.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:16.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:16.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:16.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:16.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:16.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:16.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:16.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.409 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:16.410 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.03417894399999, 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:16.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:16.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:16.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:16.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:16.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:16.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:16.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:16.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:16.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:16.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:16.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:16.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:16.420 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.421 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.421 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:16.422 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:16.422 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 239.033183603, 2023-10-30T06:28:16.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:16.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.423 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:16.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:16.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:16.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:16.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:16.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:16.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:16.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:16.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:16.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:16.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:16.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.432 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.710511032, 2023-10-30T06:28:16.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:16.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:16.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:16.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:16.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:16.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.443 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:16.444 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 231.677069547, 2023-10-30T06:28:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:16.445 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:16.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:16.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:16.455 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:16.456 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:16.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:16.458 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:16.456+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:16.558 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:16.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:16.589 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:16.590 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:16.590 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:16.599 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:16.683 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:16.682+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:16.761 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:16.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:16.768 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:16.768 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:16.768 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:16.769 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:16.769 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:16.775 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.001 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:16.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:16.861 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:16.860+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:16.881 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.096 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:16.945 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:16.945 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:16.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:16.946 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:16.946 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:16.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:16.998 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:16.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:17.013 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.238 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:17.063 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:17.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:17.082 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:17.083 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:17.083 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:17.083 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:17.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:17.110 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.325 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:17.164 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:17.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:17.260 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:17.261 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:17.262 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:17.265 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:17.264+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:17.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:17.366 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:17.364+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:17.420 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:17.420 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:17.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:17.421 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:17.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.421 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:17.422 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:17.422 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 160.44608430599999, 2023-10-30T06:28:17.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:17.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:17.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:17.423 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:17.424 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:17.424 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:17.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:17.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.425 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:17.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:17.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:17.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:17.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:17.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:17.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:17.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:17.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:17.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:17.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:17.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:17.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:17.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:17.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:17.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:17.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:17.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:17.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:17.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:17.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:17.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:17.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:17.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:17.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:17.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:17.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:17.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:17.436 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:17.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:17.437 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:17.437 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:17.437 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:17.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:17.480 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:17.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:17.568 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:17.566+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:17.571 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:17.572 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:17.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:17.668 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:17.667+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:17.767 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:17.768 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:17.768 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:17.769 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:17.768+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:17.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:17.871 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:17.869+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:17.947 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:17.947 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:17.948 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:17.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:18.001 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:17.999+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:18.073 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:18.071+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:18.128 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:18.128 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:18.128 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:18.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:18.174 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:18.172+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:18.293 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:18.292+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:18.307 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:18.308 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:18.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:18.403 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:18.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:18.476 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:18.474+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:18.540 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 245.203632742, 2023-10-30T06:28:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:18.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:18.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.559 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.559 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.560 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:18.560 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.88247755099999, 2023-10-30T06:28:18.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:18.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.560 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.561 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:18.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:18.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:18.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:18.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:18.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:18.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:18.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:18.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:18.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:18.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:18.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:18.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:18.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:18.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:18.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:18.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:18.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:18.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:18.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:18.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:18.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.878232994, 2023-10-30T06:28:18.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:18.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:18.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:18.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:18.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:18.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:18.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:18.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:18.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:18.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:18.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:18.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:18.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:18.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:18.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:18.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:18.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:18.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:18.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:18.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:18.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.593 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:18.594 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.79277456899999, 2023-10-30T06:28:18.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:18.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.594 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.594 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.595 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:18.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.595 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:18.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:18.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:18.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:18.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:18.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:18.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:18.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:18.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:18.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:18.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:18.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:18.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:18.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:18.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:18.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:18.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.610 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:18.610 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.52353135300001, 2023-10-30T06:28:18.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:18.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:18.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:18.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:18.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:18.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:18.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:18.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:18.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:18.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:18.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:18.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:18.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:18.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:18.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:18.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:18.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.522699942, 2023-10-30T06:28:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.52203281499999, 2023-10-30T06:28:18.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.643 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:18.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:18.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:18.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:18.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:18.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:18.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:18.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:18.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:18.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.657 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.49975874399999, 2023-10-30T06:28:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.658 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.659 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:18.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:18.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:18.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.49911088799999, 2023-10-30T06:28:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:18.673 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.674 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:18.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:18.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:18.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:18.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:18.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:18.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:18.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:18.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 242.49842481499999, 2023-10-30T06:28:18.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:18.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:18.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:18.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 241.42754284700001, 2023-10-30T06:28:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:18.706 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.707 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:18.724 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 241.42687888099999, 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.726 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:18.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:18.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:18.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:18.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:18.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:18.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:18.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:18.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:18.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:18.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:18.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:18.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:18.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 241.39842507200001, 2023-10-30T06:28:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:18.747 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.748 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:18.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:18.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:18.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:18.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:18.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:18.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:18.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:18.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:18.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:18.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.767 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:18.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.05665857899999, 2023-10-30T06:28:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:18.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:18.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:18.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:18.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:18.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:18.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.792 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.05590990900001, 2023-10-30T06:28:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:18.793 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.794 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:18.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:18.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:18.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:18.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:18.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:18.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:18.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:18.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.055241526, 2023-10-30T06:28:18.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.812 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:18.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:18.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:18.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:18.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:18.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:18.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:18.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.825 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.045390852, 2023-10-30T06:28:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.827 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:18.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.044717037, 2023-10-30T06:28:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:18.843 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.844 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:18.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:18.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:18.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:18.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:18.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:18.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.857 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.858 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.858 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:18.858 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.044068654, 2023-10-30T06:28:18.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:18.859 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.859 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.859 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:18.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:18.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:18.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:18.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:18.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:18.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:18.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:18.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:18.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:18.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:18.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:18.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:18.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:18.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:18.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:18.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:18.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:18.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.874 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:18.874 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:18.874 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 234.039952188, 2023-10-30T06:28:18.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:18.875 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:18.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.876 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:18.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:18.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:18.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:18.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:18.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:18.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:18.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:18.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:18.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:18.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:18.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:18.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:18.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:18.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:18.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:18.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:18.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:18.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:18.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:18.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:18.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:18.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:18.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:18.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:18.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:18.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:18.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:18.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:18.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:18.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.888 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:18.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:18.889 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:18.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:18.892 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:18.889+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:18.991 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:18.990+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:19.063 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.063 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:19.064 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:19.064 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:19.113 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:19.112+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:19.194 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:19.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:19.262 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:19.263 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:19.263 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:19.313 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:19.312+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:19.396 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:19.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:19.455 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:19.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:19.456 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:19.456 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:19.509 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:19.507+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:19.592 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.592 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:19.592 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:19.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.598 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:19.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:19.602 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:19.699 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:19.698+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:19.777 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:19.778 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:19.800 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:19.799+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:19.901 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:19.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:19.962 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:19.962 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:19.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:19.963 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:19.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.964 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:19.964 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:19.965 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 163.66993927300001, 2023-10-30T06:28:19.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:19.965 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:19.966 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:19.966 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:19.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:19.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:19.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:19.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.968 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:19.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:19.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:19.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:19.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:19.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:19.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:19.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:19.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:19.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:19.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:19.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:19.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:19.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:19.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:19.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:19.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:19.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:19.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:19.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:19.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:19.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:19.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:19.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:19.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:19.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:19.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:19.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:19.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:19.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:19.983 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:19.983 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:19.983 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:19.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:20.029 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:20.027+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:20.103 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:20.102+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:20.126 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:20.127 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:20.127 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:20.127 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:20.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:20.204 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:20.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:20.306 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:20.304+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:20.317 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:20.317 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:20.317 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:20.318 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:20.318 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:20.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:20.413 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:20.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:20.509 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:20.508+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:20.524 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:20.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:20.525 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:20.525 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:20.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:20.618 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:20.617+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:20.704 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:20.704 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:20.704 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:20.709 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:20.708+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:20.712 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:20.810 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:20.809+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:20.879 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:20.880 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:20.880 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:20.880 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:20.880 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:20.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:20.912 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:20.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:21.013 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:21.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:21.095 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:21.095 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:21.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:21.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:21.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.096 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:21.096 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:21.096 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.939598889, 2023-10-30T06:28:21.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:21.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:21.097 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:21.097 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:21.097 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:21.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:21.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:21.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:21.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:21.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:21.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:21.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:21.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:21.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:21.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:21.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:21.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.108 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.109 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:21.109 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:21.109 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 246.026425026, 2023-10-30T06:28:21.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:21.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.110 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:21.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:21.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:21.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:21.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:21.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:21.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:21.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:21.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:21.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:21.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:21.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:21.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:21.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:21.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:21.120 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.120 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.120 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 245.36624705700001, 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:21.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.122 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:21.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:21.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:21.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:21.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:21.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:21.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:21.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:21.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:21.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:21.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:21.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:21.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:21.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:21.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.133 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:21.133 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:21.133 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 245.29082385000001, 2023-10-30T06:28:21.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:21.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.134 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:21.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:21.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:21.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:21.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:21.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.145 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:21.145 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:21.145 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 245.01970071900001, 2023-10-30T06:28:21.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:21.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.146 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:21.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:21.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:21.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:21.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:21.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:21.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:21.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:21.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:21.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:21.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:21.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:21.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:21.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:21.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:21.156 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:21.157 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.98942100900001, 2023-10-30T06:28:21.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:21.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.157 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:21.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:21.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:21.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:21.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:21.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:21.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:21.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:21.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.167 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.168 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:21.168 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:21.168 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.615509925, 2023-10-30T06:28:21.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:21.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.169 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:21.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:21.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:21.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:21.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:21.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:21.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:21.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:21.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:21.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:21.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:21.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:21.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:21.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:21.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.179 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.180 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:21.180 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:21.180 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.61101122400001, 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:21.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:21.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:21.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:21.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:21.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:21.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:21.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:21.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:21.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:21.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:21.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:21.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.192 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:21.192 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:21.192 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.31940728500001, 2023-10-30T06:28:21.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:21.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.193 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:21.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:21.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:21.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:21.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:21.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:21.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:21.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:21.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:21.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:21.203 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.203 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.203 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.204 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:21.204 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:21.204 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.31346996799999, 2023-10-30T06:28:21.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:21.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.205 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:21.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:21.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:21.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:21.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:21.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:21.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:21.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:21.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:21.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:21.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:21.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:21.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:21.215 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.215 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.215 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.31217095299999, 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:21.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.217 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:21.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:21.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:21.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:21.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:21.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:21.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:21.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:21.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:21.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:21.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:21.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:21.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:21.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.227 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 244.209182145, 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.228 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:21.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:21.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:21.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:21.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:21.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:21.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:21.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:21.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:21.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:21.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:21.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:21.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:21.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:21.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.238 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.239 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.239 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:21.239 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.937278878, 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:21.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:21.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:21.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:21.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:21.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:21.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:21.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:21.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:21.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:21.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.251 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:21.252 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.92978068599999, 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:21.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:21.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:21.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:21.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:21.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:21.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:21.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:21.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:21.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:21.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:21.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:21.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:21.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:21.263 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.263 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.263 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.264 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:21.264 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:21.264 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.92911948099999, 2023-10-30T06:28:21.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:21.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:21.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:21.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:21.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:21.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:21.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:21.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:21.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:21.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:21.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:21.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:21.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:21.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:21.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.275 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.91653364300001, 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:21.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.277 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:21.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:21.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:21.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:21.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:21.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:21.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:21.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:21.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:21.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:21.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:21.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:21.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:21.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.287 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.287 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:21.288 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:21.288 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.91044142999999, 2023-10-30T06:28:21.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:21.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:21.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:21.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:21.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:21.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:21.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:21.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:21.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:21.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:21.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:21.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:21.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:21.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.298 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 243.909446089, 2023-10-30T06:28:21.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:21.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.300 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:21.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:21.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:21.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:21.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:21.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:21.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:21.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:21.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:21.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:21.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:21.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:21.309 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.310 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.310 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.310 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:21.311 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:21.311 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 236.586773518, 2023-10-30T06:28:21.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:21.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.312 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:21.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:21.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:21.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:21.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:21.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:21.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:21.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:21.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:21.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:21.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:21.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:21.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:21.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:21.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:21.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:21.322 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.322 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.322 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 236.553332033, 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:21.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:21.324 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:21.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:21.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:21.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:21.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:21.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:21.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:21.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:21.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:21.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:21.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:21.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:21.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:21.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:21.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:21.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:21.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:21.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:21.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:21.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:21.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:21.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:21.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:21.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:21.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.333 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.334 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.334 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:21.334 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:21.334 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:21.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:21.336 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:21.335+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:21.436 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:21.436+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:21.504 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:21.505 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:21.505 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:21.505 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:21.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:21.538 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:21.537+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:21.639 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:21.637+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:21.641 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:21.641 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:21.642 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:21.642 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:21.642 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:21.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:21.740 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:21.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:21.818 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:21.818 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:21.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:21.819 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:21.819 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:21.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:21.841 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:21.841+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:21.942 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:21.941+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:21.995 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:21.996 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:21.996 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:21.996 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:22.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:22.044 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:22.043+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:22.144 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:22.144+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:22.170 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:22.170 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:22.171 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:22.171 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:22.171 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:22.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:22.245 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:22.245+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:22.307 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:22.307 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:22.308 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 165.33434843500001, 2023-10-30T06:28:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:22.309 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.310 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:22.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:22.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:22.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:22.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:22.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:22.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:22.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:22.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:22.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:22.320 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:22.320 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:22.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:22.346 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:22.345+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:22.448 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:22.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:22.461 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:22.462 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:22.462 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:22.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:22.548 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:22.547+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:22.639 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:22.640 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:22.640 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:22.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:22.650 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:22.648+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:22.750 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:22.748+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:22.816 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:22.817 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:22.825 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:22.870 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:22.869+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:22.952 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:22.951+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:22.952 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:22.953 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:22.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:23.053 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:23.052+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:23.127 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:23.127 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:23.128 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:23.128 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:23.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:23.178 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:23.177+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:23.255 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:23.255+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:23.356 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:23.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 250.020177848, 2023-10-30T06:28:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:23.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.359 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:23.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:23.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:23.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:23.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:23.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:23.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:23.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:23.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:23.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.699022657, 2023-10-30T06:28:23.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.377 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:23.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:23.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:23.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:23.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:23.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:23.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:23.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:23.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:23.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:23.392 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.69477810000001, 2023-10-30T06:28:23.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:23.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:23.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:23.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:23.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:23.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:23.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:23.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:23.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:23.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:23.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:23.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:23.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:23.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:23.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:23.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:23.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.60931967499999, 2023-10-30T06:28:23.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:23.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:23.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:23.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:23.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:23.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:23.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:23.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:23.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:23.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:23.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:23.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:23.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:23.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:23.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:23.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:23.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:23.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:23.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:23.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:23.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:23.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:23.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.34007645899999, 2023-10-30T06:28:23.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:23.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.424 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:23.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:23.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:23.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:23.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:23.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:23.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:23.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:23.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:23.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:23.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:23.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:23.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:23.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:23.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:23.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:23.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:23.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:23.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:23.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:23.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.438 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.33924504800001, 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.439 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:23.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:23.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:23.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:23.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:23.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:23.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:23.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:23.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:23.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:23.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:23.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:23.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:23.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:23.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:23.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:23.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:23.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:23.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:23.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.454 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.454 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:23.454 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.338577921, 2023-10-30T06:28:23.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:23.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:23.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.456 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:23.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:23.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:23.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:23.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:23.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:23.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:23.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:23.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:23.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:23.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:23.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:23.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:23.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:23.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:23.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:23.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:23.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:23.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.468 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.469 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:23.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.31630385, 2023-10-30T06:28:23.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:23.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.470 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:23.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:23.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:23.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:23.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:23.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:23.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:23.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:23.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:23.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:23.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:23.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:23.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:23.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:23.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:23.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:23.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:23.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:23.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.485 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:23.485 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.315655994, 2023-10-30T06:28:23.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:23.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.486 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:23.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:23.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:23.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:23.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:23.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:23.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:23.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:23.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:23.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:23.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:23.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:23.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:23.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:23.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:23.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:23.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:23.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:23.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.500 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:23.500 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:23.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.314969921, 2023-10-30T06:28:23.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:23.501 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.501 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:23.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:23.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:23.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:23.518 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 246.24408795299999, 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:23.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:23.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:23.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:23.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:23.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 246.243423987, 2023-10-30T06:28:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:23.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:23.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:23.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:23.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 246.21497017799999, 2023-10-30T06:28:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.87320368499999, 2023-10-30T06:28:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:23.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.566 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:23.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:23.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:23.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:23.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:23.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:23.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:23.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.87245501500001, 2023-10-30T06:28:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.87178663200001, 2023-10-30T06:28:23.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:23.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.597 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.598 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:23.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.598 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:23.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:23.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:23.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:23.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:23.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:23.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:23.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:23.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:23.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:23.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:23.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:23.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:23.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:23.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:23.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:23.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:23.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:23.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:23.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:23.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.611 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.611 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.861935958, 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.613 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.613 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:23.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:23.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:23.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:23.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:23.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:23.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:23.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:23.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:23.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:23.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:23.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:23.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:23.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:23.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.626 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.627 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.627 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:23.628 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.861262143, 2023-10-30T06:28:23.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:23.628 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.628 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.628 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:23.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:23.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:23.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:23.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:23.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:23.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:23.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:23.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:23.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:23.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:23.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:23.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:23.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:23.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:23.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:23.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:23.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:23.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:23.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.86061376000001, 2023-10-30T06:28:23.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:23.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:23.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:23.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:23.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:23.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:23.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:23.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:23.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:23.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:23.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:23.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:23.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:23.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:23.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:23.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:23.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:23.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:23.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:23.658 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:23.659 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 238.85649729400001, 2023-10-30T06:28:23.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:23.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:23.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:23.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:23.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:23.660 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:23.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:23.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:23.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:23.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:23.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:23.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:23.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:23.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:23.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:23.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:23.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:23.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:23.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:23.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:23.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:23.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:23.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:23.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:23.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:23.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:23.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:23.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:23.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:23.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:23.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:23.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:23.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:23.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:23.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:23.674 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:23.674 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:23.356+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:23.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:23.717 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:23.716+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:23.778 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:23.778+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:23.846 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:23.846 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:23.847 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:23.847 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:23.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:23.879 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:23.878+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:23.980 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:23.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:24.022 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:24.023 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:24.023 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:24.023 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:24.023 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:24.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:24.081 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:24.080+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:24.183 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:24.182+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:24.200 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:24.200 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:24.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:24.200 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:24.201 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:24.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:24.284 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:24.282+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:24.339 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:24.340 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:24.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:24.385 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:24.383+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:24.477 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:24.478 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:24.478 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:24.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:24.487 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:24.486+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:24.587 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:24.586+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:24.658 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:24.659 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:24.660 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 168.36644078099999, 2023-10-30T06:28:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:24.661 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:24.662 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:24.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:24.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:24.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:24.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:24.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:24.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:24.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:24.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:24.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:24.674 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:24.675 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:24.675 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:24.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:24.722 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:24.720+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:24.802 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:24.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:24.818 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:24.818 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:24.819 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:24.819 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:24.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:24.890 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:24.890+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:24.991 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:24.990+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:24.994 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:24.994 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:24.995 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:24.995 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:25.092 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:25.092+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:25.130 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:25.130 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:25.131 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:25.193 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:25.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:25.268 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:25.268 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:25.269 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:25.269 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:25.294 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:25.293+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:25.395 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:25.394+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:25.403 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:25.403 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:25.404 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:25.404 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:25.496 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:25.496+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:25.582 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 253.42437388100001, 2023-10-30T06:28:25.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:25.584 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:25.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.596 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 250.51120001800001, 2023-10-30T06:28:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:25.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:25.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:25.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:25.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.85102204899999, 2023-10-30T06:28:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:25.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:25.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:25.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.77559884199999, 2023-10-30T06:28:25.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.632 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:25.633 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.504475711, 2023-10-30T06:28:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.634 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.474196001, 2023-10-30T06:28:25.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:25.657 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.10028491700001, 2023-10-30T06:28:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.658 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:25.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:25.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:25.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:25.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:25.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:25.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:25.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:25.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:25.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.669 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.09578621599999, 2023-10-30T06:28:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:25.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:25.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:25.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:25.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:25.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:25.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:25.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:25.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:25.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:25.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.681 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.804182277, 2023-10-30T06:28:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:25.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:25.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.79824496000001, 2023-10-30T06:28:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:25.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:25.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:25.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:25.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:25.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:25.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:25.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.796945945, 2023-10-30T06:28:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:25.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:25.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:25.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.69395713700001, 2023-10-30T06:28:25.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.42205387000001, 2023-10-30T06:28:25.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.414555678, 2023-10-30T06:28:25.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:25.754 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.413894473, 2023-10-30T06:28:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:25.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:25.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:25.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:25.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:25.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:25.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:25.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:25.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.40130863499999, 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:25.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:25.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:25.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:25.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:25.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:25.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.395216422, 2023-10-30T06:28:25.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:25.793 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 248.39422108100001, 2023-10-30T06:28:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.794 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 241.07154851000001, 2023-10-30T06:28:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 241.03810702499999, 2023-10-30T06:28:25.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:25.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:25.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:25.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:25.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:25.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:25.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:25.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:25.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:25.828 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:25.829 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:25.831 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:25.830+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:25.932 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:25.930+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:25.958 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:25.958 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:25.959 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:25.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:26.033 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:26.032+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:26.134 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:26.134 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:26.135 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:26.135 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:26.135 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:26.133+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:26.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:26.235 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:26.235+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:26.314 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:26.315 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:26.315 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:26.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:26.336 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:26.336+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:26.438 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:26.437+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:26.457 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:26.457 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:26.458 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:26.458 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:26.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:26.476 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:26.476 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.539 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:26.539+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:26.561 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.565 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:28:26.544+0000 11f74640 -1 mon.c@2(leader).osd e65 definitely_dead 0 2023-10-30T06:28:26.569 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6 is already out. 2023-10-30T06:28:26.596 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:26.596 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:26.597 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:26.597 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:26.597 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:26.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:26.640 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:26.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:26.673 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:26.674 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.733 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:26.734 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:26.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:26.734 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:26.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.734 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:26.735 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:26.735 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 169.76101640499999, 2023-10-30T06:28:26.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:26.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:26.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:26.736 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:26.736 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:26.736 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:26.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:26.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.737 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:26.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:26.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:26.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:26.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:26.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:26.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:26.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:26.742 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:26.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:26.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.742 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:26.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:26.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:26.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:26.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:26.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:26.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:26.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:26.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:26.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:26.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:26.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:26.745 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:26.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:26.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:26.746 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:26.746 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:26.749 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:26.750 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:26.748+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:26.757 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:26.757 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.841 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:26.841 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.849 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:26.847+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:26.881 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:26.881 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:26.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:26.924 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:26.925 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:28:26.969 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:26.968+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:27.007 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:27.008 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:28:27.051 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:27.049+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:27.053 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.053 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:27.093 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:27.093 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:28:27.152 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:27.151+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:27.221 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.221 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:27.253 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:27.252+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:27.287 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:28:27.360 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:27.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:27.382 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.383 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:27.455 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:27.454+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:27.557 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:27.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:27.561 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.561 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:27.657 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:27.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:27.738 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.738 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:27.759 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:27.758+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:27.860 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:27.859+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:27.918 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:27.918 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:27.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:27.972 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:27.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:28.005 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":32,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:28.017 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:28:28.017 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-30T06:28:28.018 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-10-30T06:28:28.057 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.057 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.059 INFO:tasks.ceph.mon.a:Started 2023-10-30T06:28:28.059 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:28:28.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:28.062 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:28.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:28.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:28.162 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:28.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:28.241 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.241 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:28.292 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:28.291+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:28.364 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:28.364+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:28.423 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.424 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:28.465 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:28.463+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:28.575 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:28.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:28.645 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.645 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:28.666 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:28.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:28.767 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:28.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:28.788 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.788 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:28.822 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":32,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:28.878 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:28.877+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:28.928 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:28.929 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:28.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:28.982 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:28.981+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:29.037 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.251 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.066 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.067 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.069 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:29.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:29.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:29.095 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.310 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:28:29.144 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.369 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.171 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:29.169+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:29.197 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.423 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.248 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.249 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.258 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:29.288 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:20.502 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.299 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:29.298+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:29.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:28:29.373 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:29.371+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:29.396 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:20.621 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:29.431 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.432 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:29.487 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:29.486+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:29.575 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.575 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.575 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:29.573+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:29.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:29.668 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:29.668 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:28:29.694 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:29.693+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:29.751 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.752 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:29.761 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:29.762 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:28:29.805 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:29.804+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:29.880 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:29.879+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:29.894 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:29.894 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:28:29.896 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:29.896 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:29.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:29.991 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:29.989+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:30.020 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:30.020 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:28:30.076 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:30.076 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:30.077 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:30.076+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:30.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:30.111 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:30.111 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:28:30.179 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:30.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:30.197 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:30.198 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:28:30.281 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:30.279+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:30.289 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:30.289 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:28:30.290 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:30.290 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:30.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:30.291 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:30.291 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:30.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:30.382 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:30.380+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:30.382 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:30.383 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:28:30.433 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:30.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:30.433 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:30.434 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:30.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:30.486 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:30.485+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:30.578 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:30.579 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:30.579 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:30.579 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:30.579 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:30.583 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:30.582+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:30.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:30.687 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:30.686+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:30.765 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:30.766 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:30.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:30.766 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:30.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.766 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:30.767 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:30.767 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 173.79314337599999, 2023-10-30T06:28:30.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:30.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:30.778 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:30.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:30.824 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:30.822+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:30.886 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:30.885+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:30.920 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:30.920 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:30.920 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:30.921 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:30.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:31.018 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:31.016+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:31.043 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:28:31.099 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:31.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:31.106 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:31.107 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:31.107 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:31.107 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:31.107 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:31.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:31.189 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:31.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:31.298 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:31.296+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:31.302 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:31.303 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:31.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:31.303 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:31.303 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:31.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:31.400 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:31.398+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:31.483 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:31.483 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:31.483 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:31.484 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:31.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:31.493 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:31.491+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:31.586 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:28:31.587 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:28:31.587 INFO:tasks.thrashosds.thrasher:Adding osd 6 2023-10-30T06:28:31.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2023-10-30T06:28:31.633 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:31.631+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:31.663 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:31.663 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:31.664 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:31.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:31.694 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:31.694+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:31.795 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:31.793+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:31.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:31.896 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:31.896+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:31.898 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:31.898 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 258.55866256000002, 2023-10-30T06:28:31.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:31.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:31.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:31.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:31.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:31.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:31.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:31.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 256.23750736900001, 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.931 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:31.932 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 256.23326281200002, 2023-10-30T06:28:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.933 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:31.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:31.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 256.14780438700001, 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.948 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.949 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:31.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:31.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:31.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:31.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:31.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:31.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:31.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.962 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.963 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.878561171, 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.964 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:31.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:31.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:31.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:31.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:31.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:31.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.87772975999999, 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:31.979 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.980 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:31.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:31.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:31.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:31.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:31.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:31.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:31.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:31.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:31.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:31.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.993 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:31.994 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.87706263300001, 2023-10-30T06:28:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:31.995 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.996 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:31.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:31.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:32.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:32.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:32.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.009 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:32.010 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.85478856200001, 2023-10-30T06:28:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.011 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.012 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:32.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:32.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:32.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:32.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:32.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:32.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:32.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.025 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:32.026 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.85414070600001, 2023-10-30T06:28:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.027 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 255.85345463300001, 2023-10-30T06:28:32.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.043 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 254.782572665, 2023-10-30T06:28:32.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.059 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:32.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:32.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.073 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 254.78190869900001, 2023-10-30T06:28:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.074 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.075 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:32.089 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 254.75345489, 2023-10-30T06:28:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.090 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:32.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.41168839700001, 2023-10-30T06:28:32.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.106 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:32.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:32.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:32.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.410939727, 2023-10-30T06:28:32.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.125 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:32.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:32.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:32.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:32.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:32.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:32.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:32.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:32.138 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.139 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.41027134399999, 2023-10-30T06:28:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.140 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:32.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:32.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:32.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:32.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.40042066999999, 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:32.155 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.156 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:32.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:32.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:32.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:32.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:32.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:32.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.39974685499999, 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.171 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:32.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:32.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:32.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:32.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:32.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:32.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:32.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:32.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:32.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:32.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:32.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.185 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.186 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.39909847199999, 2023-10-30T06:28:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.187 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 247.39498200599999, 2023-10-30T06:28:32.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:32.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:32.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:32.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:32.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:32.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:32.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:32.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:32.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:32.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:32.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:32.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:32.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:32.218 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:32.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:32.220 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:32.218+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:32.320 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:32.319+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:32.379 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:28:32.372+0000 ff6f640 -1 mon.c@2(leader).osd e68 definitely_dead 0 2023-10-30T06:28:32.390 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:32.391 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:32.391 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:32.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:28:32.400 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:32.421 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:32.421+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:32.523 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:32.521+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:32.574 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:32.575 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:32.575 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:32.575 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:32.575 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:32.576 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":32,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:32.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:32.634 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:32.633+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:32.664 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:32.664 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:28:32.753 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:32.753 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:28:32.754 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:32.751+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:32.780 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:32.780 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:32.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:32.781 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:32.781 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:32.789 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:32.825 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:32.825+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:32.839 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:32.839 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:28:32.926 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:32.925+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:32.968 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:32.968 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:28:33.007 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.008 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:33.056 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:33.057 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:28:33.058 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:33.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:33.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:28:33.155 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:33.156 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:28:33.160 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:33.158+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:33.199 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.200 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:33.240 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.6. 2023-10-30T06:28:33.247 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:33.248 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:28:33.248 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:33.247+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:33.255 INFO:tasks.thrashosds.thrasher:Added osd 6 2023-10-30T06:28:33.332 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:33.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:33.333 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:33.333 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:28:33.390 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.390 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:33.447 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:33.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:33.533 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:33.533+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:33.545 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.546 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:33.639 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:33.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:33.722 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.722 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:33.772 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:33.772+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:33.834 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:33.833+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:33.896 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:33.897 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:33.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:33.935 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:33.933+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:34.037 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.037 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.037 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:34.035+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:34.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:34.156 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:34.156+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:34.170 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.170 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:34.264 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:34.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:34.340 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:34.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:34.354 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:28:34.392 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.392 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:34.441 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:34.439+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:34.542 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:34.542+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:34.571 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.571 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:34.644 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:34.642+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:34.744 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.745 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.745 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:34.743+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:34.754 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:34.846 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:34.844+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:34.903 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:34.903 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:34.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:34.957 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:34.956+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:35.040 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:35.041 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:35.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:35.069 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:35.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:35.148 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:35.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:35.217 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:35.218 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:35.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:35.273 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:35.271+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:35.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:28:35.350 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:35.348+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:35.357 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:35.357 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:35.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:35.451 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:35.450+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:35.536 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:35.537 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:35.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:35.590 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:35.588+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:35.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:35.609 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:35.609 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:28:35.653 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:35.651+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:35.694 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:35.694 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:35.695 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:35.695 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:35.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:35.720 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:35.720 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:28:35.764 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:35.762+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:35.828 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:35.829 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:28:35.851 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:35.851 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:35.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:35.852 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:35.852 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:35.855 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:35.853+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:35.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:35.919 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:35.919 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:28:35.956 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:35.955+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:36.016 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:36.016 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:28:36.050 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:36.051 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:36.051 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:36.051 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:36.057 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:36.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:36.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:36.107 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:36.108 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:28:36.195 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:36.196 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:28:36.196 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:36.194+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:36.278 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:36.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:36.279 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:36.279 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:36.280 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:36.277+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:36.280 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:28:36.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:36.292 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":32,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":10,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:36.360 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:36.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:36.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:28:36.461 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:36.459+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:36.500 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:36.500 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:36.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:36.501 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:36.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:36.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:36.502 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 263.15096676799999, 2023-10-30T06:28:36.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:36.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:36.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.503 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:36.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:36.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:36.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:36.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:36.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:36.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:36.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:36.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:36.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:36.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:36.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:36.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:36.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:36.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:36.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:36.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:36.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:36.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:36.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:36.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:36.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.82981157699999, 2023-10-30T06:28:36.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:36.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.521 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.521 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.522 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:36.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.522 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:36.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:36.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:36.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:36.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:36.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:36.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:36.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:36.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:36.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:36.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:36.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:36.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:36.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:36.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:36.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:36.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:36.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:36.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:36.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:36.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.535 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.535 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.536 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.536 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:36.536 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.82556701999999, 2023-10-30T06:28:36.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:36.537 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.537 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.537 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.537 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.538 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.538 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:36.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.538 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:36.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:36.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:36.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:36.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:36.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:36.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:36.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:36.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:36.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:36.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:36.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:36.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:36.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:36.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:36.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:36.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:36.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:36.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.552 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.552 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:36.552 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.74010859499998, 2023-10-30T06:28:36.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:36.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.553 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.553 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.554 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.554 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:36.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.554 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:36.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:36.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:36.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:36.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:36.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:36.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:36.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:36.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:36.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:36.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:36.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:36.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:36.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:36.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:36.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:36.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:36.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:36.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.567 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.568 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.568 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:36.568 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.47086537899997, 2023-10-30T06:28:36.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:36.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.569 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.569 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.570 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.570 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:36.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.570 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:36.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:36.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:36.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:36.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:36.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:36.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:36.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:36.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:36.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:36.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:36.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:36.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:36.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:36.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:36.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:36.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:36.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:36.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.584 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.584 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:36.584 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.470033968, 2023-10-30T06:28:36.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:36.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:36.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.586 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:36.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:36.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:36.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:36.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:36.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:36.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:36.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:36.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:36.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:36.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:36.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:36.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:36.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:36.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:36.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:36.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.599 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:36.600 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.46936684100001, 2023-10-30T06:28:36.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:36.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.600 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.600 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:36.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:36.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:36.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:36.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:36.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:36.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:36.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:36.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:36.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:36.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:36.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:36.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:36.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:36.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:36.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:36.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:36.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:36.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:36.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.616 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.616 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.617 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:36.617 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.44709276999998, 2023-10-30T06:28:36.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:36.617 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.617 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.618 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.618 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:36.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:36.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:36.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:36.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:36.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:36.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:36.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:36.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:36.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:36.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:36.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:36.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:36.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:36.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:36.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:36.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:36.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:36.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:36.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:36.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:36.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:36.635 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.44644491399998, 2023-10-30T06:28:36.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:36.636 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.638 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.638 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:36.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.639 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:36.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:36.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:36.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:36.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:36.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:36.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:36.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:36.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:36.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:36.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:36.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:36.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:36.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:36.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:36.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:36.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:36.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:36.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:36.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:36.656 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.656 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.657 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:36.658 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:36.658 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 260.44575884099999, 2023-10-30T06:28:36.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:36.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.658 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:36.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:36.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:36.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:36.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:36.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:36.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:36.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:36.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:36.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:36.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:36.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:36.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:36.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:36.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:36.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:36.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:36.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:36.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:36.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:36.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:36.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.674 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:36.674 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:36.674 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 259.37487687300001, 2023-10-30T06:28:36.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:36.675 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.675 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:36.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:36.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:36.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:36.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:36.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:36.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:36.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:36.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:36.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:36.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:36.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:36.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:36.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:36.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:36.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:36.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:36.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:36.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:36.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:36.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:36.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:36.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.693 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:36.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:36.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 259.37421290700001, 2023-10-30T06:28:36.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:36.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.696 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:36.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.696 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:36.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:36.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:36.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:36.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:36.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:36.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:36.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:36.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:36.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:36.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:36.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:36.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:36.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:36.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:36.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:36.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:36.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:36.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:36.708 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.709 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.709 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.710 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:36.710 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:36.710 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 259.34575909799997, 2023-10-30T06:28:36.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:36.711 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.711 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.711 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.712 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:36.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.712 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:36.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:36.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:36.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:36.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:36.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:36.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:36.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:36.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:36.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:36.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:36.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:36.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:36.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:36.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:36.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:36.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:36.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:36.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:36.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:36.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.725 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.725 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.726 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.726 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 252.00399260500001, 2023-10-30T06:28:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.727 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:36.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:36.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:36.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:36.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:36.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:36.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:36.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:36.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:36.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:36.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:36.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:36.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:36.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:36.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:36.741 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.741 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.742 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:36.743 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 252.003243935, 2023-10-30T06:28:36.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:36.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:36.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:36.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:36.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:36.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:36.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:36.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:36.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:36.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:36.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:36.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:36.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:36.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:36.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:36.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:36.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:36.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:36.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:36.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:36.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:36.757 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.758 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.758 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 252.002575552, 2023-10-30T06:28:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:36.759 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.760 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.760 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.760 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:36.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:36.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:36.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:36.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:36.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:36.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:36.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:36.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:36.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:36.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:36.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:36.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:36.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:36.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:36.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:36.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:36.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:36.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:36.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:36.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:36.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:36.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.777 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.777 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.778 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.779 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:36.779 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 251.99272487799999, 2023-10-30T06:28:36.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:36.780 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.780 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:36.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:36.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:36.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:36.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:36.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:36.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:36.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:36.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:36.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:36.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:36.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:36.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:36.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:36.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:36.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:36.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:36.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:36.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:36.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:36.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:36.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:36.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:36.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:36.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.801 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.802 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.802 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:36.803 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 251.99205106299999, 2023-10-30T06:28:36.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:36.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.804 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.804 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.805 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.805 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:36.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.806 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:36.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:36.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:36.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:36.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:36.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:36.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:36.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:36.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:36.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:36.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:36.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:36.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:36.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:36.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:36.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:36.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:36.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:36.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:36.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:36.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:36.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.824 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.825 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.826 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.826 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:36.827 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 251.99140267999999, 2023-10-30T06:28:36.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:36.827 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.828 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.828 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.829 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.829 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:36.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.830 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:36.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:36.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:36.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:36.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:36.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:36.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:36.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:36.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:36.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:36.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:36.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:36.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:36.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:36.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:36.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:36.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.844 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:36.844 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:36.844 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 251.98728621399999, 2023-10-30T06:28:36.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:36.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:36.845 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:36.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:36.845 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:36.846 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:36.846 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:36.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.846 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:36.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:36.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:36.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:36.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:36.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:36.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:36.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:36.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:36.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:36.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:36.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:36.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:36.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:36.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:36.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:36.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:36.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:36.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:36.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:36.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:36.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:36.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:36.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:36.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:36.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:36.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:36.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:36.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:36.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:36.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:36.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:36.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:36.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:36.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:36.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:36.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:36.863 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:36.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:36.865 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.467 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:36.865 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:08.697 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:36.866 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:36.863+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:36.966 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:36.964+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:37.035 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.035 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:37.036 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:37.036 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.044 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:37.067 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:37.065+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:37.168 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:37.167+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:37.212 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.212 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:37.213 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:37.213 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:37.213 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:37.269 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:37.267+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:37.350 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:37.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:37.351 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:37.351 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:37.369 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:37.368+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:37.444 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:28:37.470 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:37.469+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:37.495 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:37.496 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:37.496 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:37.589 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:37.587+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:37.672 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:37.671+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:37.688 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:37.689 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:37.689 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:37.782 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:37.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:37.874 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:37.873+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:37.890 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:37.890 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:37.891 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 181.59366457900001, 2023-10-30T06:28:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:37.892 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.893 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:37.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:37.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:37.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:37.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:37.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:37.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:37.902 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:37.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:28:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 1.5614903600000001, 2023-10-30T06:28:37.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:28:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:37.905 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:28:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.906 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:37.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:37.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:28:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:28:37.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:28:37.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:28:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:37.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:28:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:28:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:37.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:28:37.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:28:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:28:37.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:28:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:37.916 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:37.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:37.975 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:37.974+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:38.049 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.050 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:38.050 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:38.050 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:38.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:38.076 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:38.074+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:38.177 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:38.176+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:38.232 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.232 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:38.232 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:38.233 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:38.233 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:38.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:38.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:28:38.256 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:28:38.278 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:38.277+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:38.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:28:38.370 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.370 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:38.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:38.371 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:38.371 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:38.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:38.379 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:38.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:38.480 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:38.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:38.553 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.553 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:38.553 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:38.554 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:38.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:38.581 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:38.579+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:38.608 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:38.608 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:28:38.682 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:38.681+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:38.694 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:38.695 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:28:38.729 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.730 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:38.730 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:38.730 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:38.730 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:38.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:38.780 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:38.780 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:28:38.783 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:38.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:38.873 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:38.874 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:28:38.893 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:38.891+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:38.950 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:38.951 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:38.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:38.951 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:38.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.79126113000001, 2023-10-30T06:28:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:38.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:38.964 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.965 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:38.966 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:38.966 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 263.87808726700001, 2023-10-30T06:28:38.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:38.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:38.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:38.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:38.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:38.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:38.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:38.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:38.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:38.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:38.977 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:38.978 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 263.217909298, 2023-10-30T06:28:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:38.979 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:38.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:38.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:38.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:38.982 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:38.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:38.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:38.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:38.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:38.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:38.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:38.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:38.990 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.991 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 263.14248609100002, 2023-10-30T06:28:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:38.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:38.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:38.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:38.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:38.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:38.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:38.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:38.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:39.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.002 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:39.003 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.87136296, 2023-10-30T06:28:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.004 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.014 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.84108325, 2023-10-30T06:28:39.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:39.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:39.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.026 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.46717216600001, 2023-10-30T06:28:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:39.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:39.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:39.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:39.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:39.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:39.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:39.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:39.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:39.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:39.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.46267346500002, 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.040 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:39.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:39.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:39.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:39.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:39.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:39.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:39.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:39.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:39.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:39.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:39.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:39.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:39.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:39.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:39.049 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.050 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.171069526, 2023-10-30T06:28:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.052 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:39.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:39.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:39.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:39.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:39.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:39.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:39.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:39.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:39.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:39.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:39.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.16513220899998, 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:39.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:39.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:39.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:39.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:39.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:39.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:39.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:39.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:39.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:39.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:39.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:39.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:39.073 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.073 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.074 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.074 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:39.074 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:39.075 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.16383319400001, 2023-10-30T06:28:39.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:39.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.075 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:39.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:39.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:39.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:39.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:39.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.06084438599999, 2023-10-30T06:28:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:39.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:39.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:39.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:39.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:39.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:39.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:39.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:39.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.78894111900001, 2023-10-30T06:28:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:39.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.099 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:39.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:39.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:39.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:39.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:39.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:39.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:39.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:39.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:39.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:39.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.781442927, 2023-10-30T06:28:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:39.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:39.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:39.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:39.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:39.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:39.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:39.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:39.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.121 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:39.122 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:39.122 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.78078172199997, 2023-10-30T06:28:39.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:39.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.123 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:39.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:39.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:39.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:39.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:39.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:39.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:39.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:39.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:39.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:39.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.132 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.133 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.133 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:39.133 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:39.134 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.76819588400002, 2023-10-30T06:28:39.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:39.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.134 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:39.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:39.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:39.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:39.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:39.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:39.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:39.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:39.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:39.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:39.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:39.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:39.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:39.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:39.144 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.145 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.145 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.145 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:39.146 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:39.146 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.76210367099998, 2023-10-30T06:28:39.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:39.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.147 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:39.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:39.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:39.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:39.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:39.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:39.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:39.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:39.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:39.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:39.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:39.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:39.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:39.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:39.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:39.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:39.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.157 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.157 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:39.158 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:39.158 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 261.76110833000001, 2023-10-30T06:28:39.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:39.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.159 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:39.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:39.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:39.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:39.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:39.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:39.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:39.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:39.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:39.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:39.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:39.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:39.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:39.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:39.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:39.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.169 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.169 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.169 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 254.43843575899999, 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.170 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:39.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:39.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:39.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:39.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:39.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:39.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:39.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:39.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:39.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:39.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:39.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:39.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:39.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:39.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.180 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.181 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:39.181 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:39.182 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 254.40499427399999, 2023-10-30T06:28:39.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:39.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:39.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:39.182 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:39.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:39.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:39.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:39.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:39.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:39.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:39.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:39.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:39.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:39.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:39.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:39.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:39.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:39.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:39.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:39.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:39.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:39.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:39.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:39.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:39.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:39.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:39.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:39.193 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:39.193 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:39.194 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:39.195 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:28:39.195 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:30.258 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.195 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:30.260 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.196 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:39.196 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:28:39.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:39.198 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:30.277 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.198 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:39.199 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:30.336 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.199 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:30.341 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.199 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:30.360 102049== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:39.199 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:28:39.200 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:39.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:39.272 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:39.272 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:28:39.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:39.325 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:39.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:39.369 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:39.370 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:39.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:39.396 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:39.394+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:39.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:28:39.496 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:39.495+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:39.511 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:39.511 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:39.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:39.598 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:39.596+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:39.688 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:39.688 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:39.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:39.698 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:39.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:39.800 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:39.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:39.826 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:39.826 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:39.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:39.920 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:39.918+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:40.002 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:40.000+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:40.003 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.003 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:40.102 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:40.100+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:40.107 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":32,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:40.185 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.185 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:40.239 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:40.237+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:40.256 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.136 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:40.304 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:40.302+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:40.309 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.190 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:40.326 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.326 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:40.421 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:40.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:40.506 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.506 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.507 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:40.504+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:40.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:40.569 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:12.450 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:28:40.607 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:40.605+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:40.667 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:28:40.722 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.722 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.725 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:40.723+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:40.733 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:40.819 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:40.817+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:40.905 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:40.905 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:40.909 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:40.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:40.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:41.011 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:41.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:41.083 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:41.084 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:41.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:41.112 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:41.110+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:41.213 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:41.212+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:41.228 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:41.229 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:41.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:41.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:28:41.314 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:41.313+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:41.387 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:41.387 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:41.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:41.414 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:41.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:41.515 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:41.514+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:41.542 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:41.542 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:41.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:41.616 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:41.614+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:41.639 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:41.640 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:28:41.711 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:41.711 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:41.717 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:41.716+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:41.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:41.745 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:41.746 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:28:41.819 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:41.817+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:41.833 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:41.833 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:28:41.920 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:41.918+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:41.941 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:41.942 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:28:41.943 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:41.943 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:41.943 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:41.944 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:41.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:42.021 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:42.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:42.027 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:42.027 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:28:42.104 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:42.104 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:42.104 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:42.105 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:42.105 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:42.114 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:42.121 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:42.122 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:28:42.122 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:42.120+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:42.208 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:28:42.209 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:28:42.223 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:42.222+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:42.275 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:42.275 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:42.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:42.276 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:42.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.276 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:42.276 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:42.276 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 185.97852281999999, 2023-10-30T06:28:42.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:42.277 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:42.277 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:42.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:42.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:42.278 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:42.278 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:42.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.278 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:42.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:42.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:42.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:42.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:42.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:42.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:42.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:42.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:42.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:42.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:42.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:42.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:42.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:42.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:42.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:42.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:42.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:42.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:42.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:42.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:42.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:42.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:42.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:42.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.288 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:28:42.288 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:42.288 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 5.9463486010000004, 2023-10-30T06:28:42.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:28:42.289 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:42.289 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:42.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:42.289 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:42.290 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:42.290 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:28:42.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.290 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:42.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:42.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:42.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:42.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:42.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:28:42.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:28:42.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:42.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:28:42.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:28:42.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:42.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:28:42.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:28:42.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:42.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:28:42.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:28:42.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:42.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:28:42.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:28:42.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:42.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:42.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:42.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:28:42.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:28:42.298 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:42.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:42.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:42.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:42.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:42.300 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:42.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:42.302 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:28:42.302 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:28:42.353 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:42.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:42.425 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:42.423+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:42.428 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:42.429 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:42.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:42.526 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:42.524+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:42.566 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:42.567 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:42.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:42.659 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:42.658+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:42.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:28:42.730 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:42.728+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:42.741 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:42.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:42.835 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:42.833+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:42.918 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:42.918 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:42.919 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:42.919 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:42.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:42.929 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:42.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:43.030 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:43.029+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:43.094 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:43.095 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:43.095 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:43.095 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:43.096 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:43.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:43.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:43.166 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:43.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:43.233 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:43.231+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:43.257 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:28:43.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-10-30T06:28:43.313 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:43.314 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:43.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:43.314 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:43.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.315 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:43.315 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:43.315 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.15675423900001, 2023-10-30T06:28:43.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:43.316 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:43.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:43.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.317 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:43.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:43.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:43.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:43.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:43.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:43.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:43.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:43.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:43.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:43.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:43.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:43.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.328 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:43.328 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:43.328 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 268.24358037600001, 2023-10-30T06:28:43.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:43.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.329 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:43.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:43.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:43.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:43.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:43.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:43.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:43.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:43.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:43.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:43.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:43.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:43.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:43.339 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.339 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.339 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.58340240699999, 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:43.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.341 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:43.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:43.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:43.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:43.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:43.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:43.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:43.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:43.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:43.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:43.350 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.350 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.50797920000002, 2023-10-30T06:28:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.352 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:43.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:43.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:43.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:43.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:43.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:43.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:43.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:43.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:43.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:43.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:43.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:43.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:43.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.362 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.363 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:43.363 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.236856069, 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:43.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:43.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:43.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:43.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:43.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:43.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.371 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:43.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:43.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:43.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:43.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:43.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:43.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:43.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:43.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.376 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.376 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.206576359, 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.378 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:43.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:43.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:43.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:43.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:43.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:43.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:43.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:43.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:43.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:43.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:43.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.387 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:43.388 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.83266527500001, 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:43.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:43.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:43.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:43.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:43.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:43.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:43.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:43.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:43.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:43.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:43.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:43.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:43.399 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.399 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.399 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.82816657400002, 2023-10-30T06:28:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:43.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:43.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:43.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:43.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:43.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:43.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:43.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:43.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:43.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:43.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:43.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:43.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.410 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.536562635, 2023-10-30T06:28:43.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:43.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.412 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:43.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:43.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:43.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:43.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:43.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:43.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:43.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:43.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:43.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:43.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:43.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.422 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.422 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.422 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:43.423 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:43.423 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.53062531799998, 2023-10-30T06:28:43.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:43.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:43.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:43.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:43.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:43.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:43.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:43.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.433 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.433 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.529326303, 2023-10-30T06:28:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:43.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.435 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:43.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:43.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:43.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:43.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:43.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:43.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:43.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:43.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:43.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:43.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:43.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:43.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:43.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:43.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:43.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:43.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:43.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.446 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.42633749499998, 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.448 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:43.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:43.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:43.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:43.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.453 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:43.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:43.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:43.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:43.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:43.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:43.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:43.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:43.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.457 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.458 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:43.458 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:43.458 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.15443422800001, 2023-10-30T06:28:43.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:43.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.459 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:43.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:43.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:43.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.463 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:43.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:43.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:43.466 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:43.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:43.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:43.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:43.469 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.470 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.470 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.146936036, 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.472 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:43.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:43.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:43.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:43.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:43.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:43.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:43.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.477 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.477 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:43.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:43.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.478 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:43.478 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:43.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:43.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:43.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:43.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:43.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:43.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:43.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.482 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.482 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.483 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:43.483 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:43.483 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.14627483100003, 2023-10-30T06:28:43.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:43.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.484 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:43.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:43.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:43.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:43.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:43.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:43.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:43.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:43.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:43.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.490 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.490 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:43.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:43.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:43.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:43.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.494 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.494 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:43.494 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:43.495 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.13368899300002, 2023-10-30T06:28:43.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:43.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.495 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:43.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:43.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:43.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:43.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:43.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:43.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:43.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.502 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:43.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:43.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.503 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.503 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.12759677999998, 2023-10-30T06:28:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:43.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.507 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:43.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:43.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:43.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:43.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:43.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:43.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:43.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:43.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:43.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:43.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.514 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:43.514 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:43.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.514 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:43.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:43.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:43.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:43.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.517 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.518 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:43.518 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:43.518 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.12660143900001, 2023-10-30T06:28:43.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:43.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.519 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:43.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:43.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:43.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:43.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:43.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:43.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:43.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:43.526 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:43.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:43.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:43.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.527 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.527 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:43.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:43.529 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.529 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.529 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.530 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:43.530 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:43.530 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 258.80392886800001, 2023-10-30T06:28:43.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:43.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.531 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:43.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:43.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:43.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:43.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:43.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:43.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:43.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:43.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:43.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:43.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:43.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:43.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:43.541 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.541 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.541 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.542 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:43.542 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:43.542 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 258.77048738299999, 2023-10-30T06:28:43.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:43.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:43.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:43.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:43.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:43.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:43.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:43.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:43.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:43.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:43.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:43.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:43.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:43.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:43.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:43.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:43.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:43.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:43.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:43.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:43.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:43.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:43.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:43.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:43.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:43.552 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:43.553 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:43.553 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:43.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:43.556 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:43.553+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:43.656 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:43.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:43.723 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:43.724 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:43.724 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:43.724 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:43.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:43.768 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:43.776 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:43.775+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:43.780 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:28:43.781 INFO:tasks.mon_recovery:removing mon b 2023-10-30T06:28:43.781 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T06:28:43.782 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:28:43.796 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:28:43.784+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 102047) UID: 0 2023-10-30T06:28:43.797 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:28:43.785+0000 9f64640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:28:43.858 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:43.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:43.877 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:43.877 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:43.877 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:43.877 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:43.878 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:43.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:43.972 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:43.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:44.057 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:44.058 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:44.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:44.058 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:44.058 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:44.060 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:44.058+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:44.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:44.164 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:44.162+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:44.262 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:44.261+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:44.276 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:44.276 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:44.277 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:44.277 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:44.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:44.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:28:44.314 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:35.539 102049== 2023-10-30T06:28:44.315 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:35.539 102049== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:28:44.324 INFO:tasks.ceph.mon.b.smithi112.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:28:44.363 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:44.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:44.413 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:44.414 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:44.414 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:44.414 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:44.414 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:44.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:44.464 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:44.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:44.555 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:44.556 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:44.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:44.556 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:44.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.556 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:44.557 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:44.557 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 187.581233048, 2023-10-30T06:28:44.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:44.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:44.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:44.558 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:44.558 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:44.558 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:44.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:44.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:44.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:44.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:44.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:44.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:44.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:44.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:44.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:44.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:44.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:44.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:44.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:44.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:44.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:44.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:44.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:44.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:44.566 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:44.566 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:44.567 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:44.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.567 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:28:44.567 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:44.568 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 7.5607407240000004, 2023-10-30T06:28:44.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:28:44.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:44.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:44.569 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:44.570 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:44.570 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:44.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:28:44.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.570 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:44.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:44.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:44.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:44.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:44.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:28:44.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:28:44.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:44.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:28:44.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:28:44.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:44.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:28:44.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:28:44.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:44.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:28:44.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:28:44.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:44.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:44.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:44.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:28:44.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:28:44.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:44.578 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:44.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:44.579 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:44.579 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:44.579 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:44.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:44.581 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:44.579+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:44.682 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:44.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:44.708 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:44.708 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:44.709 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:44.709 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:44.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:44.784 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:44.782+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:44.849 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:44.850 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:44.850 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:44.850 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:44.851 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:44.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:44.903 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:44.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:44.986 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:44.984+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:44.987 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:44.987 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:44.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:44.987 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:44.988 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:44.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:45.087 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:45.085+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:45.165 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:45.165 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:45.166 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:45.166 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:45.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:45.187 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:45.186+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:45.289 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:45.287+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:45.355 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:45.356 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:45.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:45.389 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:45.388+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:45.494 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:45.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:45.547 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 272.21133084000002, 2023-10-30T06:28:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.89017564900001, 2023-10-30T06:28:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.88593109200002, 2023-10-30T06:28:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.80047266700001, 2023-10-30T06:28:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.531229451, 2023-10-30T06:28:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.53039804000002, 2023-10-30T06:28:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.52973091299998, 2023-10-30T06:28:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:45.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:45.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:45.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.50745684200001, 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:45.662 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.663 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:45.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:45.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:45.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:45.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:45.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:45.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:45.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:45.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:45.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:45.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.676 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.677 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.677 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:45.677 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.50680898600001, 2023-10-30T06:28:45.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:45.678 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.678 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.678 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:45.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:45.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:45.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:45.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:45.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:45.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:45.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:45.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:45.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:45.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:45.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:45.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:45.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:45.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:45.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:45.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:45.691 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.692 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.692 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.692 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:45.693 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:45.693 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.50612291300001, 2023-10-30T06:28:45.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:45.693 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.693 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.694 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.694 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:45.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.695 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:45.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:45.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:45.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:45.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:45.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:45.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.707 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 268.43524094499998, 2023-10-30T06:28:45.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.709 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:45.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:45.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:45.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:45.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:45.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.722 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:45.723 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 268.43457697899998, 2023-10-30T06:28:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.724 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:45.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:45.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:45.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:45.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:45.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:45.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.737 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:45.738 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 268.40612317, 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:45.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:45.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:45.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:45.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:45.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:45.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.753 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.06435667699998, 2023-10-30T06:28:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.06360800700003, 2023-10-30T06:28:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:45.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:45.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:45.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:45.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:45.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:45.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.06293962400002, 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:45.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:45.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:45.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:45.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:45.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:45.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:45.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:45.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:45.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:45.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:45.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:45.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.798 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:45.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.05308895000002, 2023-10-30T06:28:45.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.800 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:45.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:45.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:45.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:45.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:45.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:45.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:45.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:45.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.05241513499999, 2023-10-30T06:28:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:45.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:45.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:45.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:45.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:45.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.05176675199999, 2023-10-30T06:28:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:45.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:45.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:45.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:45.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:45.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.845 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 261.04765028600002, 2023-10-30T06:28:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:45.846 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.847 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:45.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:45.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:45.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:45.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:45.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:45.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:45.860 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:45.860 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:45.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:45.863 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:45.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:45.963 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:45.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:45.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:45.989 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:45.990 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:45.990 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:45.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:46.064 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:46.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:46.125 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:46.125 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:46.126 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:46.126 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:46.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:46.165 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:46.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:46.266 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:46.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:46.300 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:46.301 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:46.301 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:46.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:46.367 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:46.366+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:46.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-30T06:28:46.468 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:46.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:46.481 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:46.482 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:46.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:46.575 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:46.573+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:46.658 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:46.658 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:46.659 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:46.659 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:46.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:46.709 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:46.707+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:46.771 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:46.769+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:46.839 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:46.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 190.546037985, 2023-10-30T06:28:46.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:46.842 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:46.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:46.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:46.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:46.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:46.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:46.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:46.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:46.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:46.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:46.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.852 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:28:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 10.513863766, 2023-10-30T06:28:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:28:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:46.853 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:28:46.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:46.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:46.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:28:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:28:46.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:28:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:28:46.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:28:46.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:28:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:28:46.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:28:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:46.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:28:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:28:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:46.863 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:46.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:46.864 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:46.865 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:46.910 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:46.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:46.973 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:46.972+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:47.003 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:47.004 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:47.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:47.073 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:47.072+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:47.175 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:47.173+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:47.184 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.185 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:47.185 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:47.185 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:47.185 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:47.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:47.277 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:47.275+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:47.360 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.360 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:47.361 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:47.361 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:47.370 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:47.378 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:47.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:47.478 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:47.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:47.497 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.497 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:47.498 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:47.498 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:47.507 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:47.591 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:47.590+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:47.674 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:47.766 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:47.766 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:47.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:47.768 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:47.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:47.870 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:47.868+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:47.971 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:47.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:47.974 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:47.974 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:47.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:47.975 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:47.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.975 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:47.975 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:47.976 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.81905335599998, 2023-10-30T06:28:47.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:47.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:47.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:47.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:47.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:47.977 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:47.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:47.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.978 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:47.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:47.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:47.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:47.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:47.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:47.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:47.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:47.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:47.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:47.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:47.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:47.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:47.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:47.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:47.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:47.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:47.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:47.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:47.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:47.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:47.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:47.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:47.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:47.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:47.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:47.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:47.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:47.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:47.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:47.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:47.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:47.994 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:47.994 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:47.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.995 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:47.995 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:47.995 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 272.90587949299999, 2023-10-30T06:28:47.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:47.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:47.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:47.996 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:47.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:47.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:47.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:47.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:47.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:47.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:47.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:47.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:47.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:47.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:47.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:47.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:48.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:48.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:48.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:48.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:48.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:48.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:48.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:48.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:48.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:48.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:48.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:48.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.006 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.006 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.007 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:48.007 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:48.007 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 272.24570152400003, 2023-10-30T06:28:48.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:48.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.008 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:48.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:48.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:48.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:48.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:48.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:48.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:48.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.019 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:48.019 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:48.019 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 272.170278317, 2023-10-30T06:28:48.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:48.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.020 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:48.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:48.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:48.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:48.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:48.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:48.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:48.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:48.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:48.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:48.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:48.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:48.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.030 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.89915518599997, 2023-10-30T06:28:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.032 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:48.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:48.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:48.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:48.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:48.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:48.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:48.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:48.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:48.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:48.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:48.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:48.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:48.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.041 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:48.042 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.86887547600003, 2023-10-30T06:28:48.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:48.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.043 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:48.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:48.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:48.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:48.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:48.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:48.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:48.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:48.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:48.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:48.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:48.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:48.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:48.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:48.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.053 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.054 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.054 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:48.054 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.49496439199999, 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:48.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:48.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:48.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:48.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:48.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:48.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:48.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:48.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:48.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:48.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:48.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:48.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:48.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.066 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:48.066 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:48.066 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.490465691, 2023-10-30T06:28:48.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:48.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.067 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:48.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:48.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:48.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:48.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:48.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:48.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:48.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:48.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:48.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:48.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:48.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:48.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:48.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:48.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:48.078 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.078 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:48.079 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.19886175200003, 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:48.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:48.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:48.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:48.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:48.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:48.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:48.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:48.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:48.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:48.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:48.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:48.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:48.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:48.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:48.090 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.090 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.090 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.091 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:48.091 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:48.091 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.19292443500001, 2023-10-30T06:28:48.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:48.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.092 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:48.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:48.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:48.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:48.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:48.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:48.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:48.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:48.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:48.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:48.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:48.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:48.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:48.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.102 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.102 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.103 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:48.103 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:48.103 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.19162541999998, 2023-10-30T06:28:48.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:48.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.104 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:48.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:48.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:48.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:48.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:48.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:48.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:48.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:48.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:48.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:48.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:48.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:48.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:48.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:48.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:48.114 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.114 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.114 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.115 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:48.115 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:48.115 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.08863661200002, 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:48.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:48.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:48.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:48.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:48.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:48.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:48.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:48.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:48.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:48.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:48.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:48.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:48.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:48.126 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.126 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.126 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.127 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:48.127 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:48.127 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.81673334499999, 2023-10-30T06:28:48.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:48.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.128 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:48.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:48.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:48.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:48.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:48.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:48.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:48.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:48.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:48.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:48.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.138 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.138 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.138 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:48.139 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:48.139 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.80923515299997, 2023-10-30T06:28:48.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:48.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.140 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:48.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:48.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:48.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:48.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:48.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:48.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:48.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:48.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:48.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:48.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:48.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:48.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:48.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:48.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:48.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:48.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:48.149 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.150 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:48.151 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:48.151 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.808573948, 2023-10-30T06:28:48.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:48.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.152 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:48.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:48.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:48.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:48.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:48.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:48.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:48.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:48.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:48.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:48.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:48.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:48.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:48.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:48.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.161 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.162 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:48.162 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:48.163 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.79598811, 2023-10-30T06:28:48.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:48.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.163 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:48.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:48.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:48.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:48.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:48.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:48.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:48.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:48.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:48.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:48.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:48.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.173 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:48.174 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.78989589700001, 2023-10-30T06:28:48.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:48.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.175 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:48.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:48.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:48.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:48.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:48.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.186 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:48.186 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:48.186 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 270.78890055599999, 2023-10-30T06:28:48.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:48.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.187 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:48.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:48.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:48.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:48.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:48.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:48.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:48.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:48.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:48.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:48.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:48.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:48.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:48.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:48.197 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.197 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.197 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.198 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:48.198 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:48.198 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 263.46622798499999, 2023-10-30T06:28:48.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:48.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.199 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:48.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:48.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:48.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:48.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:48.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:48.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:48.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:48.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:48.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:48.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:48.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:48.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:48.208 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.209 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.210 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:48.210 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:48.210 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 263.43278650000002, 2023-10-30T06:28:48.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:48.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:48.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:48.211 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:48.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:48.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:48.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:48.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:48.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:48.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:48.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:48.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:48.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:48.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:48.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:48.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:48.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:48.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:48.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:48.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:48.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:48.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:48.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:48.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:48.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:48.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:48.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:48.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:48.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:48.221 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:48.222 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.223 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:48.224 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:48.222+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:48.324 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:48.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:48.390 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:48.390 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:48.391 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:48.391 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.399 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:48.425 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:48.423+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:48.526 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:48.525+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:48.566 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:48.566 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:48.566 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:48.567 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:48.567 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:48.627 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:48.626+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:48.702 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:48.702 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:48.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:48.703 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:48.703 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:48.729 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:48.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:48.829 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:48.827+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:48.838 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:48.838 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:48.838 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:48.839 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:48.929 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:48.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:48.975 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:48.976 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:48.976 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:48.976 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:48.976 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:48.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:49.030 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:49.029+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:49.131 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:49.130+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:49.153 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:49.154 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:49.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:49.154 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:49.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.155 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:49.155 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:49.155 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 192.17986541799999, 2023-10-30T06:28:49.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:49.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:49.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:49.156 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:49.156 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:49.156 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:49.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:49.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.157 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:49.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:49.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:49.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:49.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:49.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:49.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:49.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:49.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:49.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:49.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:49.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:49.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:49.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:49.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:49.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:49.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:49.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:49.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:49.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:49.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:49.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:49.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:49.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:49.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:49.165 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.166 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:28:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 12.159373093999999, 2023-10-30T06:28:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:28:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:49.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.168 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:49.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:49.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:49.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:49.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:28:49.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:28:49.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:49.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:28:49.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:28:49.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:49.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:28:49.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:28:49.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:49.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:28:49.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:28:49.176 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:49.177 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:49.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:49.177 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:49.177 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:49.177 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:49.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:49.232 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:49.231+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:49.317 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:49.317 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:49.317 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:49.318 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:49.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:49.370 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:49.368+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:49.434 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:49.432+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:49.452 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:49.452 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:49.452 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:49.453 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:49.453 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:49.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:49.545 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:49.543+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:49.628 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:49.628 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:49.629 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:49.629 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:49.635 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:49.633+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:49.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:49.735 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:49.734+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:49.805 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:49.806 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:49.806 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:49.806 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:49.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:49.836 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:49.835+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:49.884 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:28:49.885 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-30T06:28:49.885 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T06:28:49.886 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:28:49.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:49.937 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:49.936+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:49.984 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:49.985 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:49.985 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:49.985 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:49.985 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:49.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:50.039 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:50.037+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:50.180 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:50.180 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:50.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:50.181 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:50.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.181 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:50.181 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:50.181 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 276.84166386200002, 2023-10-30T06:28:50.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:50.182 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.182 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.182 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.183 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.183 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:50.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.183 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:50.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:50.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:50.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:50.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:50.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:50.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:50.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:50.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:50.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:50.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:50.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:50.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:50.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:50.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:50.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:50.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:50.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:50.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:50.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:50.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:50.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:50.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:50.200 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.200 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.201 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.201 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.201 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:50.202 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.52050867100002, 2023-10-30T06:28:50.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:50.202 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.202 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.202 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.203 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.203 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:50.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.203 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:50.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:50.206 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:50.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:50.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:50.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:50.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:50.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:50.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:50.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:50.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:50.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:50.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:50.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:50.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:50.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:50.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:50.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:50.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:50.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:50.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:50.217 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.217 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.218 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.218 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:50.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.51626411400002, 2023-10-30T06:28:50.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:50.219 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.219 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.220 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:50.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:50.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:50.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:50.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:50.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:50.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:50.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:50.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:50.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:50.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:50.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:50.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:50.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:50.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:50.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:50.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:50.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:50.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:50.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:50.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:50.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:50.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.234 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.234 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.235 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.235 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:50.235 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.43080568900001, 2023-10-30T06:28:50.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.236 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:50.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:50.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:50.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.250 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:50.251 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.161562473, 2023-10-30T06:28:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.252 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:50.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:50.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:50.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:50.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:50.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:50.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:50.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.16073106200002, 2023-10-30T06:28:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:50.268 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.269 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:50.284 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.16006393499998, 2023-10-30T06:28:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:50.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:50.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:50.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:50.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:50.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.300 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:50.301 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.13778986400001, 2023-10-30T06:28:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.302 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.303 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:50.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:50.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:50.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:50.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:50.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:50.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:50.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:50.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:50.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.316 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.317 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.13714200800001, 2023-10-30T06:28:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.318 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.319 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:50.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:50.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:50.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:50.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:50.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:50.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:50.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:50.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:50.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:50.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:50.333 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.13645593500001, 2023-10-30T06:28:50.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:50.334 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.334 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.334 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.335 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.335 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:50.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.335 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:50.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:50.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:50.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:50.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:50.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:50.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:50.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:50.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:50.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:50.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:50.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:50.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:50.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:50.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:50.347 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.347 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.348 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.348 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:50.348 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:50.349 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 273.06557396699998, 2023-10-30T06:28:50.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:50.349 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.349 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.349 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.350 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.350 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:50.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.350 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:50.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:50.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:50.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:50.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:50.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:50.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:50.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:50.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:50.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:50.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:50.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:50.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:50.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:50.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:50.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:50.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:50.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:50.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:50.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:50.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:50.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:50.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:50.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:50.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.364 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.364 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.365 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:50.365 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:50.365 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 273.06491000099999, 2023-10-30T06:28:50.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:50.366 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.366 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.366 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.367 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:50.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.367 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:50.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:50.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:50.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:50.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:50.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:50.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:50.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:50.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:50.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:50.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:50.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:50.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:50.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:50.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:50.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:50.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:50.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:50.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:50.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:50.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:50.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:50.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.380 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.380 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.381 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:50.381 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:50.381 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 273.036456192, 2023-10-30T06:28:50.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:50.382 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.382 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:50.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:50.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:50.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:50.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:50.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:50.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:50.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:50.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:50.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:50.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:50.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:50.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:50.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:50.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:50.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:50.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:50.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.397 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.397 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:50.397 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.69468969899998, 2023-10-30T06:28:50.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:50.398 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.398 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.398 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:50.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:50.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:50.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:50.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:50.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:50.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:50.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:50.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:50.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:50.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:50.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:50.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:50.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:50.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:50.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:50.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:50.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:50.411 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.412 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.412 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.413 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:50.413 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.69394102899997, 2023-10-30T06:28:50.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:50.413 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.414 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.414 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.414 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:50.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.415 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:50.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:50.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:50.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:50.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:50.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:50.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:50.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:50.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:50.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:50.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:50.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:50.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:50.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:50.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:50.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:50.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:50.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:50.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:50.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.69327264600003, 2023-10-30T06:28:50.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:50.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.430 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:50.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.430 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:50.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:50.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:50.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:50.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:50.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:50.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:50.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:50.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:50.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:50.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:50.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:50.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:50.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:50.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:50.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:50.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:50.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:50.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.443 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.444 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.444 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.68342197200002, 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.446 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.446 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:50.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.446 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:50.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:50.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:50.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:50.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:50.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:50.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:50.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:50.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:50.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:50.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:50.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:50.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:50.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:50.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:50.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:50.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:50.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:50.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:50.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:50.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:50.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:50.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.459 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.460 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.460 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:50.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.68274815699999, 2023-10-30T06:28:50.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:50.461 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.461 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.462 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.462 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:50.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.463 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:50.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:50.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:50.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:50.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:50.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:50.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:50.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:50.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:50.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:50.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:50.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:50.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:50.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:50.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:50.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:50.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:50.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:50.475 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.476 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:50.477 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.68209977399999, 2023-10-30T06:28:50.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:50.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.477 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.478 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:50.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:50.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:50.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:50.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:50.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:50.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:50.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:50.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:50.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:50.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:50.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:50.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:50.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:50.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:50.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:50.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:50.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:50.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:50.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:50.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:50.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.67798330800002, 2023-10-30T06:28:50.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:50.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:50.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:50.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:50.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:50.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:50.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:50.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:50.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:50.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:50.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:50.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:50.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:50.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:50.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:50.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:50.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:50.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:50.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:50.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:50.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:50.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:50.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:50.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:50.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:50.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:50.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:50.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:50.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:50.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:50.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:50.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:50.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:50.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:50.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:50.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:50.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:50.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:50.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:50.509 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:50.510 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:50.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:50.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:50.513 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:50.511+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:50.534 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:50.619 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:50.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:50.652 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:50.652 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:50.652 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:50.652 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:50.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:50.715 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:50.714+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:50.787 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:50.787 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:50.788 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:50.788 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:50.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:50.816 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:50.814+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:50.916 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:50.915+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:50.923 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:50.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:50.924 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:50.924 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:50.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:51.017 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:51.015+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:51.062 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:51.062 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:51.062 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:51.062 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:51.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:51.117 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:51.116+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:51.218 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:51.217+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:51.241 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:51.241 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:51.241 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:51.241 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:51.241 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:51.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:51.319 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:51.318+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:51.421 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:51.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:51.422 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:51.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:51.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 195.12785511800001, 2023-10-30T06:28:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:51.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:51.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:28:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:51.434 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 15.095680899, 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:51.435 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:28:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:28:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:28:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:51.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:28:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:28:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:51.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:28:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:28:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:51.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:28:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:28:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:51.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:51.444 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:51.445 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:51.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:51.522 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:51.520+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:51.586 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:51.586 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:51.586 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:51.587 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:51.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:51.622 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:51.621+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:51.723 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:51.722+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:51.765 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:51.765 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:51.766 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:51.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:51.825 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:51.823+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:51.902 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:51.902 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:51.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:51.902 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:51.903 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:51.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:51.925 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:51.924+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:52.026 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:52.025+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:52.079 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:52.080 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:52.080 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:52.080 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:52.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:52.127 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:52.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:52.242 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:52.240+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:52.256 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:52.256 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:52.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:52.329 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:52.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:52.430 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:52.429+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:52.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:52.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.31991528100002, 2023-10-30T06:28:52.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:52.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:52.477 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:52.477 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:52.477 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.478 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:52.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:52.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:52.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:52.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:52.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:52.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:52.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:52.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:52.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:52.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:52.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:52.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:52.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:52.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:52.488 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.488 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.489 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:52.489 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:52.490 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 277.40674141800002, 2023-10-30T06:28:52.490 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:52.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.490 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:52.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:52.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:52.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:52.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:52.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:52.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:52.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:52.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:52.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:52.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:52.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:52.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:52.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:52.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:52.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:52.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:52.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.501 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.502 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:52.502 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:52.502 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 276.74656344900001, 2023-10-30T06:28:52.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:52.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.503 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.503 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:52.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:52.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:52.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:52.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:52.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:52.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:52.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:52.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:52.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:52.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:52.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.514 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:52.515 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:52.515 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 276.67114024199998, 2023-10-30T06:28:52.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:52.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:52.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:52.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:52.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:52.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:52.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:52.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:52.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.522 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:52.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:52.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:52.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:52.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.525 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.526 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:52.526 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:52.527 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 276.40001711100001, 2023-10-30T06:28:52.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:52.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.527 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:52.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:52.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:52.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:52.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:52.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:52.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:52.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:52.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:52.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:52.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:52.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:52.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:52.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:52.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:52.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.537 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.538 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:52.538 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:52.539 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 276.36973740100001, 2023-10-30T06:28:52.539 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:52.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.539 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.540 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:52.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:52.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:52.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:52.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:52.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:52.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:52.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:52.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:52.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:52.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:52.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:52.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:52.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.550 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.551 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:52.551 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:52.551 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.99582631700002, 2023-10-30T06:28:52.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:52.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.552 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:52.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:52.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:52.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:52.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:52.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:52.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:52.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:52.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:52.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:52.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:52.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:52.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:52.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:52.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:52.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:52.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:52.562 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.562 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:52.563 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.99132761599998, 2023-10-30T06:28:52.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:52.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.564 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:52.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:52.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:52.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:52.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:52.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:52.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:52.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:52.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:52.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:52.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:52.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:52.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.575 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.575 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.575 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:52.576 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:52.576 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.69972367700001, 2023-10-30T06:28:52.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:52.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:52.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:52.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:52.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:52.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:52.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:52.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:52.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:52.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:52.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:52.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:52.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:52.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:52.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.589 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.590 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:52.590 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:52.590 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.69378635999999, 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:52.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:52.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:52.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:52.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:52.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:52.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:52.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:52.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:52.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:52.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:52.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:52.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:52.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:52.601 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.601 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.602 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.602 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:52.602 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.69248734500002, 2023-10-30T06:28:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:52.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:52.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:52.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:52.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:52.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:52.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:52.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:52.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:52.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:52.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:52.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:52.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:52.613 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.614 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.614 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.614 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:52.615 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:52.615 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.589498537, 2023-10-30T06:28:52.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:52.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.616 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:52.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:52.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:52.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:52.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:52.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:52.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:52.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:52.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:52.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:52.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:52.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:52.626 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.626 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.627 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.627 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:52.627 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:52.628 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.31759527000003, 2023-10-30T06:28:52.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:52.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.628 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:52.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:52.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:52.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:52.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:52.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:52.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:52.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:52.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:52.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:52.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:52.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:52.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:52.638 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.638 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.638 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.639 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:52.639 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:52.639 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.31009707800001, 2023-10-30T06:28:52.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:52.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.640 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:52.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:52.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:52.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:52.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:52.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:52.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.646 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:52.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:52.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:52.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:52.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:52.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:52.649 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.650 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.651 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:52.651 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:52.651 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.30943587299998, 2023-10-30T06:28:52.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:52.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.652 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:52.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:52.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:52.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:52.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:52.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:52.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:52.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:52.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:52.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:52.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:52.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:52.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:52.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:52.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:52.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:52.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:52.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.662 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.663 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:52.663 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:52.663 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.29685003499998, 2023-10-30T06:28:52.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:52.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.664 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:52.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:52.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:52.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:52.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:52.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:52.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:52.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:52.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:52.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:52.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:52.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:52.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:52.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:52.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:52.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:52.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:52.674 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.675 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.675 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:52.676 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:52.676 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.29075782199999, 2023-10-30T06:28:52.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:52.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.677 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:52.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:52.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:52.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:52.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:52.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:52.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:52.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:52.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:52.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:52.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:52.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:52.687 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.687 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.687 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.688 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:52.688 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:52.688 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.28976248100003, 2023-10-30T06:28:52.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:52.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:52.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:52.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:52.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:52.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:52.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:52.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:52.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:52.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:52.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:52.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:52.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:52.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:52.699 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.699 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.700 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.700 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:52.700 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.96708991000003, 2023-10-30T06:28:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.701 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:52.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:52.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:52.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:52.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:52.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:52.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:52.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:52.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:52.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:52.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:52.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:52.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:52.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:52.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:52.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.712 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 267.933648425, 2023-10-30T06:28:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:52.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:52.714 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:52.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:52.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:52.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:52.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:52.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:52.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:52.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:52.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:52.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:52.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:52.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:52.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:52.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:52.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:52.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:52.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:52.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:52.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:52.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:52.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:52.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:52.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:52.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:52.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:52.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:52.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:52.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:52.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:52.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.724 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:52.725 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:52.725 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:52.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:52.727 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:52.725+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:52.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-30T06:28:52.827 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:52.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:52.852 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:52.852 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:52.853 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:52.853 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:52.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:52.927 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:52.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:52.989 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:52.989 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:52.990 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:52.990 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:52.990 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:52.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:53.028 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:53.027+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:53.125 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:53.126 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:53.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:53.126 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:53.126 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:53.129 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:53.128+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:53.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:53.229 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:53.228+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:53.303 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:53.304 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:53.304 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:53.304 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:53.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:53.331 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:53.329+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:53.432 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:53.430+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:53.479 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:53.479 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:53.479 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:53.480 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:53.480 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:53.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:53.532 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:53.531+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:53.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:28:53.617 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:53.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 196.64340593399999, 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:53.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:53.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:53.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:53.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:53.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:53.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:53.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.629 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 16.622913610000001, 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:53.630 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:28:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.631 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:28:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:28:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:53.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:28:53.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:28:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:28:53.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:28:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:28:53.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:53.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:53.641 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:53.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:53.742 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:53.740+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:53.775 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:53.775 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:53.776 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:53.776 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:53.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:53.869 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:53.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:53.944 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:53.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:53.954 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:53.954 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:53.955 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:53.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:54.045 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:54.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:54.134 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:54.135 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:54.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:54.136 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:54.136 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:54.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:54.145 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:54.144+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:54.257 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:54.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:54.272 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:54.272 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:54.273 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:54.273 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:54.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:54.366 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:54.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:54.448 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:54.448 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:54.448 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:54.448 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:54.449 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:54.449 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:54.447+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:54.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:54.549 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:54.548+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:54.651 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:54.649+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:54.680 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:54.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 281.34258199300001, 2023-10-30T06:28:54.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:54.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:54.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:54.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:54.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:54.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:54.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:54.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:54.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.702 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.703 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.02142680200001, 2023-10-30T06:28:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.704 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:54.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:54.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:54.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:54.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:54.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:54.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:54.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:54.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.01718224500001, 2023-10-30T06:28:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:54.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:54.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:54.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:54.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:54.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:54.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:54.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:54.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.93172382, 2023-10-30T06:28:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:54.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:54.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:54.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:54.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.662480604, 2023-10-30T06:28:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:54.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:54.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:54.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:54.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:54.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:54.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:54.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:54.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.66164919300002, 2023-10-30T06:28:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:54.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.769 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:54.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:54.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:54.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:54.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.782 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.783 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.66098206599997, 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.638707995, 2023-10-30T06:28:54.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.800 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:54.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:54.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:54.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.638060139, 2023-10-30T06:28:54.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:54.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 278.63737406600001, 2023-10-30T06:28:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:54.830 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.831 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.832 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.833 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:54.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:54.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:54.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:54.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 277.56649209800003, 2023-10-30T06:28:54.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:54.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:54.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:54.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:54.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:54.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:54.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:54.862 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 277.56582813199998, 2023-10-30T06:28:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:54.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:54.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:54.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:54.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:54.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:54.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:54.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:54.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 277.53737432299999, 2023-10-30T06:28:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:54.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.19560782999997, 2023-10-30T06:28:54.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.894 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:54.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:54.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:54.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:54.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:54.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:54.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:54.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:54.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.908 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.19485916000002, 2023-10-30T06:28:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.909 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.910 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:54.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:54.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:54.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:54.923 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.924 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.19419077700002, 2023-10-30T06:28:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.925 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:54.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.18434010300001, 2023-10-30T06:28:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:54.940 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:54.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:54.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:54.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:54.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:54.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:54.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:54.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.18366628799998, 2023-10-30T06:28:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:54.955 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.956 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:54.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:54.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:54.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:54.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:54.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:54.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:54.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:54.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:54.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.18301790499999, 2023-10-30T06:28:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:54.971 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.972 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:54.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:54.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:54.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:54.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:54.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:54.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:54.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.986 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.987 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 270.17890143900001, 2023-10-30T06:28:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:54.988 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:54.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:54.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:54.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:54.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:54.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:54.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:54.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:54.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:54.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:54.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:54.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:55.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:55.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:55.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:55.003 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:55.004 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:55.004 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:55.007 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:55.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:55.107 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:55.106+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:55.178 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.178 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:55.179 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:55.179 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:55.208 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:55.207+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:55.310 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:55.308+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:55.356 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:55.357 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:55.357 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:28:55.411 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:55.409+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:55.493 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.493 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:55.494 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:55.494 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:28:55.544 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:55.542+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:55.612 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:55.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:55.668 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.668 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:55.669 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:55.669 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:28:55.713 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:55.712+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:55.814 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:55.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:55.844 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:55.844 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:55.845 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:55.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:55.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:28:55.914 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:55.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:55.983 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:55.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:28:55.984 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 199.689644649, 2023-10-30T06:28:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:28:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:55.985 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:28:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.986 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:55.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:28:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:55.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:28:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:28:55.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:28:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:28:55.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:28:55.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:28:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:28:55.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:28:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:28:55.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:28:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:55.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:28:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:28:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:55.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:28:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:55.996 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 19.65747043, 2023-10-30T06:28:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:28:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:28:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:55.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:28:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:28:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:55.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:55.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:28:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:56.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:28:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:28:56.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:28:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:28:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:28:56.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:28:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:56.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:28:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:28:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:56.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:28:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:28:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:28:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:28:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:56.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:56.008 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:56.008 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:56.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:28:56.054 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:56.052+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:56.115 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:56.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:56.152 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:56.152 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:56.152 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:56.152 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:56.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:28:56.216 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:56.215+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:56.318 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:56.316+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:56.326 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:56.326 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:56.327 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:56.327 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:56.327 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:56.335 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:28:56.418 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:56.417+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:56.461 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:56.461 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:56.462 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:56.462 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:56.462 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:56.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:28:56.519 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:56.518+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:56.614 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:56.615 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:56.615 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:56.615 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:56.620 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:56.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:56.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:28:56.721 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:56.720+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:56.792 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:56.793 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:56.793 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:56.793 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:56.793 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:56.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:28:56.843 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:56.841+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:56.923 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:56.922+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:57.006 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:57.007 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:57.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:57.007 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:57.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.008 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:28:57.008 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:57.008 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 284.85122968600001, 2023-10-30T06:28:57.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:28:57.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:57.009 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:57.009 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:28:57.009 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:57.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:28:57.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.010 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:57.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:28:57.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:28:57.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:28:57.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:28:57.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:28:57.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:28:57.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:28:57.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:28:57.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:28:57.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:28:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:28:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:28:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:28:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:28:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:28:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:28:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.020 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.021 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:28:57.021 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:57.021 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 281.93805582300001, 2023-10-30T06:28:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:28:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:28:57.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:28:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:28:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:28:57.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:28:57.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:28:57.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:28:57.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:28:57.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:28:57.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:28:57.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:28:57.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:28:57.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:28:57.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:28:57.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:28:57.033 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.033 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.034 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.034 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:28:57.034 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:57.035 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 281.277877854, 2023-10-30T06:28:57.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:28:57.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.035 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:57.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:28:57.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:28:57.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:28:57.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:28:57.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:28:57.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:28:57.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:28:57.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:28:57.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:28:57.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:28:57.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:28:57.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:28:57.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:28:57.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:28:57.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:28:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:28:57.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:28:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.047 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:28:57.047 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:57.047 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 281.20245464700002, 2023-10-30T06:28:57.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:28:57.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.048 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:57.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:28:57.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:28:57.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:28:57.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:28:57.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:28:57.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:28:57.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:28:57.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:28:57.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:28:57.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:28:57.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:28:57.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:28:57.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:28:57.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:28:57.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:28:57.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.058 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.058 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.059 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:28:57.059 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.931331516, 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:57.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:28:57.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:28:57.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:28:57.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:28:57.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:28:57.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:28:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:28:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:28:57.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:28:57.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:28:57.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:28:57.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:28:57.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:28:57.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:28:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:28:57.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.071 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.071 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.071 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:28:57.072 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:57.072 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.901051806, 2023-10-30T06:28:57.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:28:57.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.073 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:57.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:28:57.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:28:57.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:28:57.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:28:57.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:28:57.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:28:57.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:28:57.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:28:57.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:28:57.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:28:57.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:28:57.082 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.082 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.083 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:28:57.083 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:57.084 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.52714072200001, 2023-10-30T06:28:57.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:28:57.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.084 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:57.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:28:57.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:28:57.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:28:57.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:28:57.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:28:57.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:28:57.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:28:57.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:28:57.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:28:57.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:28:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:28:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:28:57.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:28:57.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:28:57.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:28:57.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.096 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:28:57.096 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:57.096 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.52264202100002, 2023-10-30T06:28:57.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:28:57.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.097 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:57.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:28:57.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:28:57.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:28:57.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:28:57.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:28:57.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:28:57.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:28:57.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:28:57.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:28:57.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:28:57.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:28:57.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:28:57.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:28:57.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:28:57.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:28:57.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:28:57.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:28:57.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.107 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.108 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:28:57.108 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:57.108 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.231038082, 2023-10-30T06:28:57.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:57.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:28:57.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:28:57.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:28:57.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:28:57.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:28:57.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:28:57.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:28:57.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:28:57.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:28:57.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:28:57.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:28:57.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:28:57.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:28:57.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:28:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:28:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:28:57.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:28:57.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.119 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:57.120 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.22510076499998, 2023-10-30T06:28:57.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:28:57.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.121 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:57.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:28:57.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:28:57.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:28:57.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:28:57.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:28:57.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:28:57.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:28:57.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:28:57.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:28:57.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:28:57.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:28:57.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:28:57.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:28:57.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:28:57.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.131 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.132 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:28:57.132 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.22380175000001, 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:57.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:28:57.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:28:57.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:28:57.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:28:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:28:57.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:28:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:28:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:28:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:28:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:28:57.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:28:57.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:28:57.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:28:57.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.143 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.144 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:28:57.144 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:57.144 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.12081294199999, 2023-10-30T06:28:57.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:28:57.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.145 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:57.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:28:57.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:28:57.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:28:57.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:28:57.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:28:57.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:28:57.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:28:57.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:28:57.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:28:57.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:28:57.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:28:57.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:28:57.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:28:57.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:28:57.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:28:57.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:28:57.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:28:57.155 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.156 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.156 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:28:57.157 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:57.157 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.84890967500002, 2023-10-30T06:28:57.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:28:57.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.158 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:57.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:28:57.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:28:57.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:28:57.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:28:57.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:28:57.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:28:57.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:28:57.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:28:57.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:28:57.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:28:57.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:28:57.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:28:57.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:28:57.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:28:57.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:28:57.167 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.168 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.168 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:28:57.169 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:57.169 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.841411483, 2023-10-30T06:28:57.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:28:57.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.170 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:57.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:28:57.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:28:57.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:28:57.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:28:57.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:28:57.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:28:57.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:28:57.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:28:57.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:28:57.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:28:57.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:28:57.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:28:57.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:28:57.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:28:57.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:28:57.179 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.180 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.84075027799997, 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.182 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:57.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:28:57.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:28:57.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:28:57.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:28:57.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:28:57.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:28:57.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:28:57.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:28:57.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:28:57.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:28:57.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:28:57.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:28:57.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:28:57.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:28:57.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.192 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.193 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:28:57.193 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:57.193 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.82816444000002, 2023-10-30T06:28:57.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:28:57.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.194 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:57.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:28:57.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:28:57.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:28:57.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:28:57.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:28:57.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:28:57.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:28:57.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:28:57.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:28:57.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:28:57.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:28:57.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:28:57.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:28:57.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:28:57.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:28:57.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:28:57.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:28:57.204 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.205 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:28:57.206 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:57.206 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.82207222699998, 2023-10-30T06:28:57.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:28:57.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:57.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:28:57.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:28:57.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:28:57.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:28:57.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:28:57.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:28:57.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:28:57.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:28:57.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:28:57.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:28:57.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:28:57.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:28:57.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:28:57.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:28:57.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.217 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:28:57.218 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:57.218 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 279.82107688600001, 2023-10-30T06:28:57.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:28:57.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.219 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:57.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:28:57.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:28:57.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:28:57.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:28:57.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:28:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:28:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:28:57.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:28:57.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:28:57.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:28:57.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:28:57.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:28:57.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.228 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:28:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:57.229 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 272.49840431500002, 2023-10-30T06:28:57.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:28:57.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.230 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:57.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:28:57.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:28:57.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:28:57.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:28:57.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:28:57.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:28:57.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:28:57.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:28:57.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:28:57.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:28:57.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:28:57.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:28:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:28:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:28:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:28:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.241 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:28:57.241 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:57.241 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 272.46496282999999, 2023-10-30T06:28:57.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:28:57.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:57.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:57.242 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:57.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:57.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:57.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:57.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:28:57.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:57.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:57.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:28:57.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:28:57.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:57.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:28:57.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:28:57.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:57.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:28:57.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:28:57.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:57.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:28:57.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:28:57.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:28:57.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:28:57.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:28:57.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:28:57.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:28:57.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:28:57.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:57.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:57.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:57.253 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:57.253 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:57.254 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:28:57.255 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:28:57.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:28:57.355 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:57.354+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:57.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:57.424 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:57.424 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:57.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:28:57.477 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:57.476+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:57.558 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:57.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:57.559 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:57.559 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:57.559 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:57.560 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:57.560 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:57.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:28:57.658 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:57.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:57.735 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:57.735 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:57.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:57.736 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:28:57.736 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:57.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:28:57.759 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:57.758+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:57.860 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:57.858+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:57.913 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:57.913 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:57.913 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:28:57.914 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:57.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:28:57.960 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:57.959+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:58.061 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:58.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:58.088 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:58.088 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:28:58.088 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:28:58.088 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:28:58.089 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:58.098 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:28:58.162 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:28:58.161+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:28:58.230 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:28:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:28:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:28:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:28:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:28:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 201.255326881, 2023-10-30T06:28:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:28:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:28:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:28:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:28:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:28:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:28:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:28:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:28:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:28:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:28:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:28:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:28:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 21.234834556999999, 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:28:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:28:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:28:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:28:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:28:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:28:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:28:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:28:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:28:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:28:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:28:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:28:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:28:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:28:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:28:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:28:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:28:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:28:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:28:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:28:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:28:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:28:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:28:58.253 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:28:58.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:28:58.262 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:28:58.261+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:28:58.364 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:58.362+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:58.394 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:58.395 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:58.395 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:58.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:28:58.465 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:58.463+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:58.566 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:58.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:58.592 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:58.592 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:58.592 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:58.592 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:58.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:58.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:28:58.667 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:58.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:58.766 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:28:58.767 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:58.769 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:58.767+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:58.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:28:58.869 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:58.868+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:58.947 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:58.947 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:58.947 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:28:58.947 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:58.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:28:58.998 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:28:58.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:28:59.071 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:59.070+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:59.089 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:28:59.105 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:28:59.106 INFO:tasks.mon_recovery:causing some monitor log activity 2023-10-30T06:28:59.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2023-10-30T06:28:59.153 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:59.153 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:28:59.154 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:28:59.154 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:28:59.154 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:59.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:28:59.172 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:59.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:59.273 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:59.272+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:28:59.378 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:28:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:28:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:28:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:28:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:28:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 286.023618951, 2023-10-30T06:28:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:28:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:28:59.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:59.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:28:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:28:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:28:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:28:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:28:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:28:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:28:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:28:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:28:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:28:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:28:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:28:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:28:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:28:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:28:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:28:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:28:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:28:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:28:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:28:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:28:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:28:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:28:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:28:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:28:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:28:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.70246376, 2023-10-30T06:28:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:28:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:28:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:28:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:28:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:28:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:28:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:28:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:28:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:28:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:28:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:28:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:28:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:28:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:28:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:28:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:28:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:28:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:28:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:28:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:28:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:28:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:28:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:28:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.69821920300001, 2023-10-30T06:28:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:28:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:28:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:28:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:28:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:28:59.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:28:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:28:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:28:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:28:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:28:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:28:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:28:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:28:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:28:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:28:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:28:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:28:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:28:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:28:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:59.430 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.61276077799999, 2023-10-30T06:28:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:28:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:28:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:28:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:28:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:28:59.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:28:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:28:59.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:28:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:28:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:28:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:28:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:28:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:28:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:28:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:28:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:28:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:28:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:28:59.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:28:59.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:28:59.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:28:59.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.451 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.451 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.452 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.452 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:59.452 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.34351756199999, 2023-10-30T06:28:59.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:28:59.452 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.453 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.453 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.453 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:28:59.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.454 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:59.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:28:59.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:28:59.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:28:59.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:28:59.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:28:59.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:28:59.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:28:59.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:28:59.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:28:59.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:28:59.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:28:59.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:28:59.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:28:59.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:28:59.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:28:59.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:28:59.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:28:59.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:28:59.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:28:59.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.470 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:59.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.34268615100001, 2023-10-30T06:28:59.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:28:59.471 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.471 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.472 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:28:59.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:59.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:28:59.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:28:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:28:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:28:59.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:28:59.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:28:59.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:28:59.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:28:59.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:28:59.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:28:59.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:28:59.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:28:59.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:28:59.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:28:59.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:28:59.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:28:59.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:28:59.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.486 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:59.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.34201902400002, 2023-10-30T06:28:59.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:28:59.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.487 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:28:59.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:28:59.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:28:59.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:28:59.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:28:59.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:28:59.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:28:59.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:28:59.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:28:59.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:28:59.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:28:59.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:28:59.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:28:59.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:28:59.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:28:59.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:28:59.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:28:59.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:28:59.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:28:59.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:28:59.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.501 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:59.502 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.319744953, 2023-10-30T06:28:59.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:28:59.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.503 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:28:59.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.503 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:59.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:28:59.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:28:59.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:28:59.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:28:59.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:28:59.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:28:59.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:28:59.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:28:59.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:28:59.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:28:59.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:28:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:28:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:28:59.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:28:59.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:28:59.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:28:59.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:28:59.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:28:59.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:28:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:28:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.517 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:59.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.319097097, 2023-10-30T06:28:59.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:28:59.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.518 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:28:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:28:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:28:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:28:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:28:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:28:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:28:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:28:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:28:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:28:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:28:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:28:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:28:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:28:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:28:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:28:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:28:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:28:59.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:28:59.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:28:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:28:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:28:59.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:59.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 283.318411024, 2023-10-30T06:28:59.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:28:59.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.534 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:28:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:28:59.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:28:59.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:28:59.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:28:59.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:28:59.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:28:59.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:28:59.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:28:59.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:28:59.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:28:59.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:28:59.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:28:59.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:28:59.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:28:59.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:28:59.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:28:59.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:28:59.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:28:59.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:28:59.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:59.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:59.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.24752905600002, 2023-10-30T06:28:59.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:28:59.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.551 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:28:59.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:59.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:28:59.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:28:59.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:28:59.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:28:59.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:28:59.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:28:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:28:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:28:59.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:28:59.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:28:59.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:28:59.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:28:59.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:28:59.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:28:59.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:28:59.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:28:59.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:28:59.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:28:59.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:28:59.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:28:59.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:28:59.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:28:59.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:28:59.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.565 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:59.565 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:59.565 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.24686509000003, 2023-10-30T06:28:59.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:28:59.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.566 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.566 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.567 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.567 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:28:59.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.567 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:59.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:28:59.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:28:59.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:28:59.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:28:59.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:28:59.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:28:59.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:28:59.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:28:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:28:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:28:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:28:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:28:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:28:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:28:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:28:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:28:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:28:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:28:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:28:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:28:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:28:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:28:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 282.21841128099999, 2023-10-30T06:28:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:28:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:28:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:28:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:28:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:28:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:28:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:28:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:28:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:28:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:28:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:28:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:28:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:28:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:28:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:28:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:28:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:28:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:28:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:28:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:28:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:28:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:28:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:28:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.87664478800002, 2023-10-30T06:28:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:28:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:28:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:28:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:28:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:28:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:28:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:28:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:28:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:28:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:28:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:28:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:28:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:28:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:28:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:28:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:28:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:28:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:28:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:28:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.87589611800001, 2023-10-30T06:28:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:28:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:28:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:59.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:28:59.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:28:59.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:28:59.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:28:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:28:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:28:59.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:28:59.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:28:59.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:28:59.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:28:59.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:28:59.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:28:59.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:28:59.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:28:59.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:28:59.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:28:59.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:28:59.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:28:59.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:28:59.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.633 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.634 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.634 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:59.634 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.87522773500001, 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.635 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:28:59.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.636 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:59.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:28:59.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:28:59.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:28:59.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:28:59.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:28:59.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:28:59.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:28:59.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:28:59.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:28:59.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:28:59.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:28:59.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:28:59.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:28:59.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:28:59.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:28:59.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:28:59.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:28:59.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:28:59.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:28:59.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.649 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.650 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.650 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:59.650 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.865377061, 2023-10-30T06:28:59.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:28:59.650 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.651 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.651 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.651 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:28:59.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.652 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:59.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:28:59.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:28:59.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:28:59.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:28:59.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:28:59.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:28:59.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:28:59.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:28:59.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:28:59.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:28:59.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:28:59.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:28:59.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:28:59.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:28:59.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:28:59.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:28:59.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:28:59.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:28:59.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:28:59.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:28:59.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.665 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.666 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:59.666 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.86470324599998, 2023-10-30T06:28:59.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:28:59.666 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.666 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.667 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.667 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:28:59.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:59.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:28:59.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:28:59.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:28:59.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:28:59.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:28:59.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:28:59.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:28:59.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:28:59.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:28:59.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:28:59.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:28:59.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:28:59.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:28:59.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:28:59.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:28:59.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:28:59.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:28:59.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:28:59.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:28:59.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:28:59.680 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.680 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.681 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.681 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.681 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:59.682 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.86405486299998, 2023-10-30T06:28:59.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:28:59.682 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.682 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.682 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.683 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.683 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:28:59.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.683 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:59.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:28:59.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:28:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:28:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:28:59.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:28:59.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:28:59.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:28:59.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:28:59.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:28:59.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:28:59.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:28:59.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:28:59.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:28:59.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:28:59.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:28:59.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:28:59.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:28:59.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:28:59.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:28:59.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:28:59.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:28:59.696 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.696 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.697 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.697 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:28:59.697 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:59.698 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.85993839700001, 2023-10-30T06:28:59.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:28:59.698 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:28:59.698 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:28:59.698 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:28:59.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:28:59.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:28:59.699 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:28:59.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.699 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:28:59.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:28:59.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:59.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:28:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:28:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:28:59.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:28:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:28:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:28:59.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:28:59.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:28:59.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:28:59.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:28:59.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:28:59.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:28:59.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:28:59.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:28:59.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:28:59.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:28:59.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:28:59.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:28:59.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:28:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:28:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:28:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:28:59.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:28:59.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:28:59.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:28:59.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:28:59.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:28:59.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:28:59.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:28:59.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:28:59.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:28:59.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:28:59.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.712 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.712 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.713 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:28:59.713 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:28:59.713 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:28:59.713 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:28:59.376+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:28:59.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:28:59.715 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:28:59.716 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.716 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:28:59.716 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.717 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:28:59.717 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.719 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:28:59.718+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:28:59.755 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:28:59.756 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.820 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:28:59.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:28:59.859 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:28:59.859 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.902 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:28:59.902 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:28:59.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:28:59.951 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:28:59.952 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:28:59.952 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:28:59.950+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:00.021 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:00.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:00.034 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:00.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.045 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:00.117 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:00.117 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:00.122 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:00.121+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:00.223 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:00.222+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:00.230 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.230 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:00.324 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:00.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:00.399 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:29:00.401 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.401 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.410 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2023-10-30T06:29:00.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:00.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-10-30T06:29:00.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2023-10-30T06:29:00.452 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:00.449+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:00.537 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:00.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:00.566 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.567 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:00.626 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:00.625+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:00.728 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:00.726+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:00.754 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.754 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.763 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:00.829 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:00.827+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:00.899 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:00.900 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:00.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:00.930 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:00.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:01.031 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:01.030+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:01.075 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.075 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.084 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:01.169 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:01.167+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:01.234 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:01.232+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:01.251 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.252 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:01.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2023-10-30T06:29:01.345 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:01.344+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:01.429 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.429 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.435 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:01.433+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:01.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:01.456 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2023-10-30T06:29:01.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-10-30T06:29:01.536 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:01.535+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:01.605 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.606 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:01.659 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:01.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:01.738 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:01.737+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:01.743 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.743 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:01.839 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:01.838+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:01.918 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:01.919 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:01.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:01.972 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:01.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:02.040 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:02.039+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:02.054 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:02.054 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:02.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:02.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:02.176 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:02.174+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:02.229 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:02.229 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:02.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:02.241 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:02.240+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:02.342 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:02.341+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:02.364 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:02.364 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:02.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:02.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:29:02.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2023-10-30T06:29:02.485 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:02.483+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:02.517 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:02.517 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.544 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:02.543+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:02.558 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:02.558 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:02.564 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2023-10-30T06:29:02.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:02.578 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-10-30T06:29:02.622 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:02.622 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.651 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:02.650+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:02.735 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:02.735 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:02.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:02.746 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:02.744+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:02.762 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:02.763 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.847 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:02.847 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.848 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:02.846+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:02.914 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:02.914 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:02.914 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:02.915 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:02.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:02.943 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:02.944 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:02.988 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:02.986+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:03.035 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:03.035 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:03.060 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:03.059+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:03.117 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:03.118 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:03.127 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:03.127 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:03.127 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:03.128 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:03.128 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:03.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:03.150 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:03.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:03.251 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:03.250+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:03.314 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:03.315 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:03.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:03.367 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:03.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:03.452 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:03.453 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:03.453 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:03.451+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:03.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:03.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2023-10-30T06:29:03.554 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:03.553+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:03.643 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:29:03.644 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2023-10-30T06:29:03.655 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:03.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:03.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-10-30T06:29:03.664 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:03.664 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:03.665 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:03.665 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:03.665 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:03.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:03.755 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:03.754+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:03.819 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:35.699 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:03.825 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:35.706 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:03.856 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:03.855+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:03.858 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:00:35.739 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:03.915 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:03.915 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:03.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:03.916 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:03.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.916 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:03.916 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:03.917 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 290.57605248099998, 2023-10-30T06:29:03.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:03.917 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.917 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.917 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:03.918 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:03.918 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:03.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.918 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:03.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:03.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:03.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:03.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:03.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:03.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:03.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:03.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:03.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:03.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:03.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:03.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:03.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:03.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:03.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:03.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:03.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:03.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:03.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:03.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:03.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:03.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:03.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:03.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:03.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:03.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:03.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:03.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:03.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:03.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:03.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:03.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:03.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:03.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:03.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:03.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:03.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 288.25489728999997, 2023-10-30T06:29:03.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:03.936 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:03.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:03.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:03.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:03.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:03.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:03.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:03.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:03.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:03.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:03.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:03.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:03.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:03.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:03.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:03.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:03.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:03.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:03.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:03.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:03.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:03.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:03.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:03.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:03.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:03.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:03.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:03.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:03.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:03.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:03.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:03.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:03.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:03.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.950 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:03.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.951 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:03.951 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:03.951 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 288.25065273299998, 2023-10-30T06:29:03.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:03.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:03.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:03.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:03.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.953 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:03.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:03.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:03.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:03.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:03.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:03.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:03.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:03.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:03.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:03.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:03.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:03.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:03.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:03.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:03.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:03.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:03.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:03.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:03.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:03.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:03.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:03.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:03.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:03.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:03.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:03.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:03.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:03.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:03.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:03.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:03.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:03.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 288.16519430800003, 2023-10-30T06:29:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:03.967 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:03.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:03.968 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:03.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:03.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:03.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:03.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:03.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:03.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:03.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:03.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:03.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:03.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:03.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:03.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:03.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:03.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:03.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:03.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:03.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:03.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:03.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:03.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:03.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:03.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:03.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:03.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:03.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:03.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:03.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:03.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:03.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:03.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:03.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.89595109200002, 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.983 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:03.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:03.984 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:03.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.985 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:03.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:03.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:03.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:03.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:03.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:03.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:03.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:03.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:03.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:03.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:03.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:03.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:03.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:03.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:03.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:03.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:03.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:03.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:03.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:03.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:03.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.89511968099998, 2023-10-30T06:29:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:03.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:04.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:04.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.894452554, 2023-10-30T06:29:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:04.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:04.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:04.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:04.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:04.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:04.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:04.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:04.028 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.87217848300003, 2023-10-30T06:29:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:04.030 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.031 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.045 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:04.046 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.87153062700003, 2023-10-30T06:29:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.047 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.87084455399997, 2023-10-30T06:29:04.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.063 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.079 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:04.080 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 286.79996258599999, 2023-10-30T06:29:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.081 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.095 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:04.096 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 286.79929862, 2023-10-30T06:29:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.097 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.111 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:04.112 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 286.77084481100002, 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.113 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:04.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:04.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:04.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.42907831799999, 2023-10-30T06:29:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.129 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:04.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:04.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:04.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:04.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:04.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:04.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:04.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:04.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:04.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:04.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:04.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:04.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:04.142 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.143 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.144 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:04.144 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.42832964799999, 2023-10-30T06:29:04.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:04.144 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.144 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.145 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.145 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:04.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.146 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:04.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:04.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:04.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:04.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:04.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:04.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:04.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:04.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:04.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:04.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:04.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:04.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:04.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:04.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:04.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:04.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:04.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:04.158 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.159 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.42766126499998, 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.160 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.161 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.161 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:04.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.161 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:04.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:04.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:04.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:04.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:04.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:04.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:04.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:04.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:04.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:04.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:04.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:04.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.41781059099998, 2023-10-30T06:29:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.177 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:04.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.177 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:04.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:04.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:04.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:04.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:04.182 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:04.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.182 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.182 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:04.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:04.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.183 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.183 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:04.184 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:04.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.184 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.184 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:04.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:04.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:04.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:04.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:04.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:04.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:04.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:04.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.190 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.190 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.191 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.191 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:04.191 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.41713677600001, 2023-10-30T06:29:04.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:04.191 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.192 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.192 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.192 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:04.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.193 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:04.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:04.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:04.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:04.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:04.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:04.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:04.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:04.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:04.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:04.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:04.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:04.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:04.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:04.205 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.41648839300001, 2023-10-30T06:29:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.207 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:04.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:04.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:04.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:04.211 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:04.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.212 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.212 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:04.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:04.213 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.214 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.214 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.214 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:04.214 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:04.214 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.215 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:04.215 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:04.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:04.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:04.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:04.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:04.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.218 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:04.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:04.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:04.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:04.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:04.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:04.221 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.221 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.222 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.222 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:04.222 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 279.41237192699998, 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:04.223 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:04.224 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:04.224 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:04.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.224 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:04.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:04.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:04.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:04.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:04.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:04.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:04.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:04.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:04.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:04.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:04.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:04.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:04.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:04.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:04.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:04.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:04.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:04.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:04.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:04.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:04.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:04.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:04.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:04.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:04.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:04.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:04.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.237 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:04.237 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:04.238 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:04.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:04.240 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:55.311 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:04.241 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:55.315 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:04.241 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:55.334 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:04.246 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:04.243+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:04.340 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:04.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:04.411 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:04.412 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:04.412 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:04.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:04.466 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:04.464+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:04.542 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:04.541+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:04.595 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:04.595 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:04.596 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:04.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:04.644 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:04.643+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:04.759 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:04.758+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:04.774 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:04.775 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:04.775 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:04.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:04.845 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:04.844+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:04.947 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:04.945+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:04.950 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:04.950 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:04.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:05.048 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:05.047+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:05.126 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:05.127 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:05.133 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:05.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:05.176 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:05.175+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:05.250 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:05.249+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:05.312 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:05.312 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 209.01773763599999, 2023-10-30T06:29:05.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:05.314 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:05.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:05.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 28.985563417000002, 2023-10-30T06:29:05.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:05.325 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:05.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:05.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:05.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:05.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:05.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:05.335 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:05.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:05.351 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:05.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:05.368 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:05.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.451 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:05.452 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.452 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:05.451+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:05.474 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:05.475 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:05.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:05.496 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2023-10-30T06:29:05.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2023-10-30T06:29:05.541 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:05.542 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.553 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:05.551+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:05.614 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:05.615 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:05.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:05.650 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:05.650 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:29:05.664 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:05.663+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:05.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2023-10-30T06:29:05.749 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:05.749 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.754 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:05.754+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:05.795 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:05.795 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:05.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:05.835 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:05.835 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.856 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:05.854+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:05.942 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:05.942 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:05.952 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:05.952 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:05.963 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:05.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:05.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:06.033 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:06.034 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:06.058 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:06.057+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:06.131 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.132 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:06.182 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:06.180+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:06.261 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:06.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:06.304 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.305 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:06.361 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:06.360+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:06.440 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.441 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:06.462 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:06.461+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:06.562 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:06.562+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:06.618 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.619 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:06.664 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:06.663+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:06.668 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2023-10-30T06:29:06.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-10-30T06:29:06.725 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:29:06.764 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.764 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.765 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:06.763+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:06.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:06.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2023-10-30T06:29:06.865 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:06.864+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:06.943 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:06.944 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:06.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:06.998 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:06.996+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:07.067 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:07.066+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:07.119 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.120 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:07.171 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:07.170+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:07.269 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:07.268+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:07.295 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.296 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:07.370 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:07.369+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:07.436 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.436 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:07.471 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:07.470+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:07.572 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:07.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:07.578 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.578 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:07.674 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:07.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:07.754 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.755 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:07.775 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:07.774+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:07.829 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2023-10-30T06:29:07.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-10-30T06:29:07.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2023-10-30T06:29:07.908 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:07.908 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:07.909 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:07.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:07.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:07.976 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:07.976+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:08.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:08.077 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:08.077+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:08.108 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:08.108 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:08.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:08.179 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:08.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:08.249 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:08.249 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:08.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:08.300 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:08.299+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:08.381 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:08.381+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:08.440 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.482 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:08.481+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:08.483 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:08.483 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:08.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:08.544 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:08.545 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.582 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:08.582+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:08.664 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:08.665 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.689 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:08.689+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:08.717 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:08.717 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:08.717 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:08.718 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:08.718 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:08.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:08.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:29:08.785 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:08.785+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:08.790 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:08.791 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:08.879 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.885 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:08.885+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:08.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:08.904 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:08.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:08.963 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2023-10-30T06:29:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:08.969 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:08.979 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-10-30T06:29:08.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:29:09.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2023-10-30T06:29:09.019 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:09.018+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:09.060 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:09.074 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:09.075 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:09.075 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:09.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:09.087 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:09.086+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:09.161 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:09.162 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:09.188 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:09.188+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:09.265 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:09.266 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:09.267 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:09.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:09.289 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:09.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:09.390 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:09.390+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:09.416 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 213.11879969899999, 2023-10-30T06:29:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 33.086625480000002, 2023-10-30T06:29:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:09.439 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:09.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:09.491 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:09.490+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:09.573 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:09.573 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:09.573 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:09.574 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:09.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:09.592 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:09.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:09.693 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:09.692+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:09.750 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:09.750 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:09.750 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:09.750 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:09.751 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:09.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:09.805 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:09.804+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:09.889 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:09.890 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:09.890 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:09.895 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:09.894+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:09.898 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:09.899 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:29:09.900 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:29:09.912 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2023-10-30T06:29:09.307530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221768,"kb_used_data":5596,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752952,"statfs":{"total":773094113280,"available":772867022848,"internally_reserved":0,"allocated":5730304,"data_stored":3320111,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000618"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.737032+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:29:03.737032+0000","last_peered":"2023-10-30T06:29:03.737032+0000","last_clean":"2023-10-30T06:29:03.737032+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:29:03.737032+0000","last_undegraded":"2023-10-30T06:29:03.737032+0000","last_fullsized":"2023-10-30T06:29:03.737032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.911685+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:29:02.911685+0000","last_peered":"2023-10-30T06:29:02.911685+0000","last_clean":"2023-10-30T06:29:02.911685+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:29:02.911685+0000","last_undegraded":"2023-10-30T06:29:02.911685+0000","last_fullsized":"2023-10-30T06:29:02.911685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.568467+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:29:05.568467+0000","last_peered":"2023-10-30T06:29:05.568467+0000","last_clean":"2023-10-30T06:29:05.568467+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:29:05.568467+0000","last_undegraded":"2023-10-30T06:29:05.568467+0000","last_fullsized":"2023-10-30T06:29:05.568467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.558305+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:29:02.558305+0000","last_peered":"2023-10-30T06:29:02.558305+0000","last_clean":"2023-10-30T06:29:02.558305+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:29:02.558305+0000","last_undegraded":"2023-10-30T06:29:02.558305+0000","last_fullsized":"2023-10-30T06:29:02.558305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.732899+0000","last_change":"2023-10-30T06:28:34.904618+0000","last_active":"2023-10-30T06:29:03.732899+0000","last_peered":"2023-10-30T06:29:03.732899+0000","last_clean":"2023-10-30T06:29:03.732899+0000","last_became_active":"2023-10-30T06:28:34.898267+0000","last_became_peered":"2023-10-30T06:28:34.898267+0000","last_unstale":"2023-10-30T06:29:03.732899+0000","last_undegraded":"2023-10-30T06:29:03.732899+0000","last_fullsized":"2023-10-30T06:29:03.732899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:12.089156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.668969+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:29:02.668969+0000","last_peered":"2023-10-30T06:29:02.668969+0000","last_clean":"2023-10-30T06:29:02.668969+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:29:02.668969+0000","last_undegraded":"2023-10-30T06:29:02.668969+0000","last_fullsized":"2023-10-30T06:29:02.668969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.729529+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:29:03.729529+0000","last_peered":"2023-10-30T06:29:03.729529+0000","last_clean":"2023-10-30T06:29:03.729529+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:29:03.729529+0000","last_undegraded":"2023-10-30T06:29:03.729529+0000","last_fullsized":"2023-10-30T06:29:03.729529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.667265+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:29:02.667265+0000","last_peered":"2023-10-30T06:29:02.667265+0000","last_clean":"2023-10-30T06:29:02.667265+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:29:02.667265+0000","last_undegraded":"2023-10-30T06:29:02.667265+0000","last_fullsized":"2023-10-30T06:29:02.667265+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.563550+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:29:05.563550+0000","last_peered":"2023-10-30T06:29:05.563550+0000","last_clean":"2023-10-30T06:29:05.563550+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:29:05.563550+0000","last_undegraded":"2023-10-30T06:29:05.563550+0000","last_fullsized":"2023-10-30T06:29:05.563550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411391,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":719623,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.114000000000001,"5min":10.382999999999999,"15min":10.367000000000001},"min":{"1min":4.8570000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":41.07,"5min":41.07,"15min":41.07},"last":7.7409999999999997},{"interface":"front","average":{"1min":11.855,"5min":11.709,"15min":11.958},"min":{"1min":6.0679999999999996,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":44.597999999999999,"5min":46.798000000000002,"15min":46.798000000000002},"last":8.891}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.618,"5min":12.460000000000001,"15min":13.352},"min":{"1min":6.2060000000000004,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":23.099,"5min":52.552,"15min":52.552},"last":8.1029999999999998},{"interface":"front","average":{"1min":13.275,"5min":14.632,"15min":14.975},"min":{"1min":8.3219999999999992,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":25.076000000000001,"5min":55.585999999999999,"15min":55.585999999999999},"last":11.816000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":15.544,"5min":18.59,"15min":21.657},"min":{"1min":9.6660000000000004,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":23.303999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":15.771000000000001},{"interface":"front","average":{"1min":14.035,"5min":17.509,"15min":20.884},"min":{"1min":8.2300000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":31.236000000000001,"5min":124.075,"15min":124.075},"last":12.601000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":13.478999999999999,"5min":13.329000000000001,"15min":13.558999999999999},"min":{"1min":7.1319999999999997,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":32.243000000000002,"5min":51.649000000000001,"15min":51.649000000000001},"last":9.9900000000000002},{"interface":"front","average":{"1min":16.407,"5min":17.34,"15min":17.779},"min":{"1min":8.8810000000000002,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":25.788,"5min":44.973999999999997,"15min":44.973999999999997},"last":13.694000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":15.675000000000001,"5min":17.978000000000002,"15min":20.027000000000001},"min":{"1min":8.7349999999999994,"5min":7.351,"15min":7.351},"max":{"1min":28.606999999999999,"5min":123.754,"15min":123.754},"last":14.321},{"interface":"front","average":{"1min":16.48,"5min":18.271999999999998,"15min":21.477},"min":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"max":{"1min":35.601999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":10.672000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":17.298999999999999,"5min":19.186,"15min":20.699999999999999},"min":{"1min":9.3279999999999994,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":33.847000000000001,"5min":66.400999999999996,"15min":66.400999999999996},"last":20.018000000000001},{"interface":"front","average":{"1min":17.619,"5min":18.045999999999999,"15min":19.254000000000001},"min":{"1min":8.1929999999999996,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":43.159999999999997,"5min":53.149000000000001,"15min":53.149000000000001},"last":22.119}]},{"osd":7,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":17.652000000000001,"5min":19.731000000000002,"15min":20.919},"min":{"1min":11.337999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":27.016999999999999,"5min":51.375,"15min":51.375},"last":17.085999999999999},{"interface":"front","average":{"1min":18.718,"5min":20.405000000000001,"15min":21.198},"min":{"1min":10.598000000000001,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":38.033999999999999,"5min":69.991,"15min":69.991},"last":15.066000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476800,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":321783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.73,"5min":12.119,"15min":13.843},"min":{"1min":5.3339999999999996,"5min":4.7779999999999996,"15min":4.7779999999999996},"max":{"1min":26.062000000000001,"5min":94.343000000000004,"15min":94.343000000000004},"last":6.0519999999999996},{"interface":"front","average":{"1min":10.679,"5min":12.59,"15min":14.699},"min":{"1min":5.0049999999999999,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":22.806000000000001,"5min":122.054,"15min":122.054},"last":7.1070000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.141,"5min":10.465999999999999,"15min":10.699},"min":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"max":{"1min":21.786000000000001,"5min":21.786000000000001,"15min":21.786000000000001},"last":19.745999999999999},{"interface":"front","average":{"1min":15.263999999999999,"5min":21.969000000000001,"15min":29.899000000000001},"min":{"1min":5.9989999999999997,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":36.935000000000002,"5min":387.15600000000001,"15min":387.15600000000001},"last":20.484999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.308999999999999,"5min":15.284000000000001,"15min":18.398},"min":{"1min":6.6799999999999997,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":16.988,"5min":156.90299999999999,"15min":156.90299999999999},"last":22.667999999999999},{"interface":"front","average":{"1min":15.432,"5min":17.199000000000002,"15min":19.454999999999998},"min":{"1min":11.369999999999999,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":22.655000000000001,"5min":143.578,"15min":143.578},"last":23.745000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":18.914999999999999,"15min":20.946999999999999},"min":{"1min":7.7999999999999998,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":27.901,"5min":141.18799999999999,"15min":141.18799999999999},"last":21.300000000000001},{"interface":"front","average":{"1min":18.728000000000002,"5min":21.913,"15min":24.251000000000001},"min":{"1min":9.1099999999999994,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":35.975999999999999,"5min":161.32900000000001,"15min":161.32900000000001},"last":29.498999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.367000000000001,"5min":13.390000000000001,"15min":14.493},"min":{"1min":7.835,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":22.731000000000002,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.5069999999999997},{"interface":"front","average":{"1min":15.497,"5min":15.002000000000001,"15min":15.468},"min":{"1min":7.6379999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"max":{"1min":24.989999999999998,"5min":58.613999999999997,"15min":58.613999999999997},"last":25.407}]},{"osd":5,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":13.66,"5min":15.308,"15min":17.079999999999998},"min":{"1min":8.4019999999999992,"5min":7.383,"15min":7.383},"max":{"1min":24.629999999999999,"5min":71.757000000000005,"15min":71.757000000000005},"last":10.41},{"interface":"front","average":{"1min":18.666,"5min":19.806999999999999,"15min":21.109999999999999},"min":{"1min":9.532,"5min":9.532,"15min":9.532},"max":{"1min":29.719999999999999,"5min":80.119,"15min":80.119},"last":24.709}]},{"osd":7,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":17.417000000000002,"15min":18.966000000000001},"min":{"1min":9.9329999999999998,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":23.734999999999999,"5min":81.247,"15min":81.247},"last":11.686999999999999},{"interface":"front","average":{"1min":16.643999999999998,"5min":17.164999999999999,"15min":18.446999999999999},"min":{"1min":8.6310000000000002,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":31.199000000000002,"5min":70.989000000000004,"15min":70.989000000000004},"last":27.321999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476800,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":10.869999999999999,"15min":10.996},"min":{"1min":4.9450000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":48.792000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":12.890000000000001},{"interface":"front","average":{"1min":11.84,"5min":10.802,"15min":10.439},"min":{"1min":7.0129999999999999,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":28.616,"5min":28.616,"15min":28.616},"last":6.915}]},{"osd":1,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":13.914999999999999,"5min":19.594000000000001,"15min":24.416},"min":{"1min":7.46,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":33.198,"5min":165.76400000000001,"15min":165.76400000000001},"last":15.276},{"interface":"front","average":{"1min":13.18,"5min":19.954999999999998,"15min":25.370999999999999},"min":{"1min":7.8289999999999997,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":21.282,"5min":190.34200000000001,"15min":190.34200000000001},"last":12.006}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":16.754999999999999,"15min":17.870000000000001},"min":{"1min":8.548,"5min":7.96,"15min":7.96},"max":{"1min":34.350000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":13.701000000000001},{"interface":"front","average":{"1min":15.375999999999999,"5min":15.984,"15min":16.794},"min":{"1min":8.109,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":47.140999999999998,"5min":47.140999999999998,"15min":47.140999999999998},"last":17.533000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.747,"5min":18.550999999999998,"15min":19.93},"min":{"1min":11.375,"5min":9.8979999999999997,"15min":9.8979999999999997},"max":{"1min":44.091000000000001,"5min":44.091000000000001,"15min":44.091000000000001},"last":16.076000000000001},{"interface":"front","average":{"1min":12.366,"5min":11.036,"15min":11.420999999999999},"min":{"1min":5.9400000000000004,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":45.231000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.2550000000000008}]},{"osd":5,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.690999999999999,"5min":16.504999999999999,"15min":18.606999999999999},"min":{"1min":11.563000000000001,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":49.784999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":19.428000000000001},{"interface":"front","average":{"1min":13.282999999999999,"5min":17.286999999999999,"15min":19.628},"min":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":28.132999999999999,"5min":58.576000000000001,"15min":58.576000000000001},"last":8.8559999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":15.202,"5min":22.263999999999999,"15min":26.846},"min":{"1min":8.4260000000000002,"5min":8.4260000000000002,"15min":8.4260000000000002},"max":{"1min":29.905000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":9.9979999999999993},{"interface":"front","average":{"1min":16.574000000000002,"5min":14.781000000000001,"15min":15.256},"min":{"1min":9.5060000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":34.969000000000001,"5min":40.128999999999998,"15min":40.128999999999998},"last":10.561}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.837,"5min":18.343,"15min":18.105},"min":{"1min":12.76,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":50.722000000000001,"5min":50.722000000000001,"15min":50.722000000000001},"last":21.396999999999998},{"interface":"front","average":{"1min":18.157,"5min":23.681000000000001,"15min":27.637},"min":{"1min":11.978999999999999,"5min":10.025,"15min":10.025},"max":{"1min":29.071999999999999,"5min":170.232,"15min":170.232},"last":19.818000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":9.9510000000000005,"5min":17.408999999999999,"15min":24.341999999999999},"min":{"1min":5.4710000000000001,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":27.117000000000001,"5min":258.66199999999998,"15min":258.66199999999998},"last":15.102},{"interface":"front","average":{"1min":10.667,"5min":15.890000000000001,"15min":21.158000000000001},"min":{"1min":5.3499999999999996,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":28.109999999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":11.819000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.951000000000001,"5min":14.247999999999999,"15min":15.622999999999999},"min":{"1min":7.468,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":27.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":13.709},{"interface":"front","average":{"1min":12.977,"5min":13.169,"15min":14.282},"min":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":6.6070000000000002},"max":{"1min":21.780999999999999,"5min":37.390999999999998,"15min":37.390999999999998},"last":19.134}]},{"osd":3,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.968999999999999,"5min":14.577,"15min":17.742999999999999},"min":{"1min":6.8899999999999997,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":23.518999999999998,"5min":183.38900000000001,"15min":183.38900000000001},"last":6.8019999999999996},{"interface":"front","average":{"1min":16.148,"5min":18.513000000000002,"15min":21.548999999999999},"min":{"1min":7.6399999999999997,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.055,"5min":189.374,"15min":189.374},"last":15.715}]},{"osd":4,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.247999999999999,"5min":17.015000000000001,"15min":20.311},"min":{"1min":8.3019999999999996,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":16.420999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":9.2040000000000006},{"interface":"front","average":{"1min":16.623000000000001,"5min":16.381,"15min":18.870000000000001},"min":{"1min":8.9849999999999994,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":36.784999999999997,"5min":167.67599999999999,"15min":167.67599999999999},"last":18.524000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":18.126999999999999,"5min":21.486000000000001,"15min":25.535},"min":{"1min":10.58,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":24.699000000000002,"5min":215.80500000000001,"15min":215.80500000000001},"last":16.844999999999999},{"interface":"front","average":{"1min":17.141999999999999,"5min":20.536000000000001,"15min":25.065999999999999},"min":{"1min":8.8520000000000003,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":33.207999999999998,"5min":187.863,"15min":187.863},"last":13.087}]},{"osd":6,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.503,"5min":20.372,"15min":24.965},"min":{"1min":11.701000000000001,"5min":10.32,"15min":10.32},"max":{"1min":28.986999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":7.7969999999999997},{"interface":"front","average":{"1min":20.504999999999999,"5min":23.777000000000001,"15min":28.158000000000001},"min":{"1min":12.313000000000001,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":33.987000000000002,"5min":275.92200000000003,"15min":275.92200000000003},"last":20.614999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":19.542999999999999,"5min":21.734999999999999,"15min":25.396000000000001},"min":{"1min":9.4580000000000002,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":35.898000000000003,"5min":170.43600000000001,"15min":170.43600000000001},"last":17.850999999999999},{"interface":"front","average":{"1min":18.407,"5min":19.861999999999998,"15min":22.312000000000001},"min":{"1min":8.5280000000000005,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.297000000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":10.759}]}]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":9.4700000000000006,"5min":10.786,"15min":10.725},"min":{"1min":4.9829999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":25.042999999999999,"5min":56.704000000000001,"15min":56.704000000000001},"last":10.776},{"interface":"front","average":{"1min":10.782999999999999,"5min":11.834,"15min":11.882999999999999},"min":{"1min":5.8559999999999999,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":26.018999999999998,"5min":57.805,"15min":57.805},"last":10.196999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":10.135,"5min":13.997,"15min":18.033000000000001},"min":{"1min":6.5469999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":17.658000000000001,"5min":127.95399999999999,"15min":127.95399999999999},"last":8.7260000000000009},{"interface":"front","average":{"1min":13.305,"5min":13.374000000000001,"15min":14.372},"min":{"1min":7.4390000000000001,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":35.310000000000002,"5min":35.834000000000003,"15min":35.834000000000003},"last":12.269}]},{"osd":3,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.784000000000001,"5min":16.899000000000001,"15min":20.733000000000001},"min":{"1min":8.6069999999999993,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":18.431000000000001,"5min":99.790999999999997,"15min":99.790999999999997},"last":20.157},{"interface":"front","average":{"1min":15.534000000000001,"5min":15.677,"15min":16.369},"min":{"1min":10.504,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":43.860999999999997,"5min":43.860999999999997,"15min":43.860999999999997},"last":21.922999999999998}]},{"osd":4,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":14.250999999999999,"5min":14.385999999999999,"15min":15.895},"min":{"1min":8.2260000000000009,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":51.802,"5min":51.802,"15min":51.802},"last":7.1559999999999997},{"interface":"front","average":{"1min":18.021000000000001,"5min":19.768000000000001,"15min":23.504999999999999},"min":{"1min":10.582000000000001,"5min":7.6689999999999996,"15min":7.6689999999999996},"max":{"1min":40.000999999999998,"5min":103.705,"15min":103.705},"last":14.912000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.994,"5min":15.896000000000001,"15min":17.991},"min":{"1min":9.1519999999999992,"5min":7.8330000000000002,"15min":7.8330000000000002},"max":{"1min":17.576000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":12.898999999999999},{"interface":"front","average":{"1min":19.388000000000002,"5min":21.844999999999999,"15min":26.18},"min":{"1min":9.3119999999999994,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":40.598999999999997,"5min":98.707999999999998,"15min":98.707999999999998},"last":18.882000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.209,"5min":19.010999999999999,"15min":20.006},"min":{"1min":11.734,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":51.018999999999998,"5min":55.631,"15min":55.631},"last":16.119},{"interface":"front","average":{"1min":16.334,"5min":19.213000000000001,"15min":21.271000000000001},"min":{"1min":9.907,"5min":9.7159999999999993,"15min":9.7159999999999993},"max":{"1min":48.860999999999997,"5min":54.656999999999996,"15min":54.656999999999996},"last":17.809000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.692,"5min":19.841000000000001,"15min":21.363},"min":{"1min":12.294,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":53.314,"5min":53.314,"15min":53.314},"last":22.963999999999999},{"interface":"front","average":{"1min":18.108000000000001,"5min":20.082000000000001,"15min":22.216000000000001},"min":{"1min":10.589,"5min":10.589,"15min":10.589},"max":{"1min":37.427999999999997,"5min":45.130000000000003,"15min":45.130000000000003},"last":15.539999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":310709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.51,"5min":12.632999999999999,"15min":15.106999999999999},"min":{"1min":4.944,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":40.948,"5min":131.477,"15min":131.477},"last":16.724},{"interface":"front","average":{"1min":11.792999999999999,"5min":13.177,"15min":15.381},"min":{"1min":6.4649999999999999,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":40.159999999999997,"5min":129.386,"15min":129.386},"last":7.0339999999999998}]},{"osd":1,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":10.955,"5min":10.936999999999999,"15min":10.975},"min":{"1min":6.8940000000000001,"5min":6.3390000000000004,"15min":6.3390000000000004},"max":{"1min":18.311,"5min":25.774000000000001,"15min":25.774000000000001},"last":10.638},{"interface":"front","average":{"1min":15.336,"5min":17.414999999999999,"15min":18.797000000000001},"min":{"1min":12.584,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":19.048999999999999,"5min":91.295000000000002,"15min":91.295000000000002},"last":17.68}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.068,"5min":14.746,"15min":17.061},"min":{"1min":6.5110000000000001,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":21.138999999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":12.663},{"interface":"front","average":{"1min":12.34,"5min":14.003,"15min":14.423999999999999},"min":{"1min":7.3540000000000001,"5min":5.992,"15min":5.992},"max":{"1min":22.401,"5min":50.450000000000003,"15min":50.450000000000003},"last":18.460999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":14.092000000000001,"5min":16.588999999999999,"15min":19.309000000000001},"min":{"1min":7.2750000000000004,"5min":6.758,"15min":6.758},"max":{"1min":22.478999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":8.2750000000000004},{"interface":"front","average":{"1min":16.396999999999998,"5min":17.763999999999999,"15min":20.045999999999999},"min":{"1min":11.826000000000001,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":22.414999999999999,"5min":124.684,"15min":124.684},"last":15.577}]},{"osd":4,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":14.701000000000001,"5min":13.255000000000001,"15min":13.677},"min":{"1min":10.32,"5min":6.6050000000000004,"15min":6.6050000000000004},"max":{"1min":21.858000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":13.587},{"interface":"front","average":{"1min":17.369,"5min":16.047000000000001,"15min":15.75},"min":{"1min":10.175000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":28.701000000000001,"5min":28.701000000000001,"15min":28.701000000000001},"last":20.657}]},{"osd":5,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":11.826000000000001,"5min":16.309999999999999,"15min":19.253},"min":{"1min":6.9550000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":20.228000000000002,"5min":125.792,"15min":125.792},"last":11.827999999999999},{"interface":"front","average":{"1min":16.962,"5min":16.856000000000002,"15min":16.956},"min":{"1min":12.593999999999999,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":28.417999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":19.93}]},{"osd":6,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.372,"5min":22.959,"15min":29.097999999999999},"min":{"1min":13.268000000000001,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":30.768999999999998,"5min":327.29599999999999,"15min":327.29599999999999},"last":21.893999999999998},{"interface":"front","average":{"1min":13.772,"5min":22.376000000000001,"15min":30.071000000000002},"min":{"1min":8.5779999999999994,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":25.986999999999998,"5min":323.82400000000001,"15min":323.82400000000001},"last":9.7859999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":708549,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":8.4239999999999995,"5min":11.695,"15min":13.956},"min":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":16.850000000000001,"5min":68.513999999999996,"15min":68.513999999999996},"last":8.5830000000000002},{"interface":"front","average":{"1min":9.6069999999999993,"5min":12.859999999999999,"15min":15.647},"min":{"1min":6.8209999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":14.297000000000001,"5min":88.816000000000003,"15min":88.816000000000003},"last":9.7910000000000004}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":12.551,"5min":15.057,"15min":16.774999999999999},"min":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":21.32,"5min":87.515000000000001,"15min":87.515000000000001},"last":32.015999999999998},{"interface":"front","average":{"1min":13.327,"5min":15.451000000000001,"15min":17.158999999999999},"min":{"1min":8.8840000000000003,"5min":6.21,"15min":6.21},"max":{"1min":22.943000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":36.521000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":14.614000000000001,"5min":16.452000000000002,"15min":17.725999999999999},"min":{"1min":6.5259999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":27.128,"5min":113.943,"15min":113.943},"last":25.742000000000001},{"interface":"front","average":{"1min":10.391,"5min":12.885999999999999,"15min":14.608000000000001},"min":{"1min":6.2889999999999997,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.457999999999998,"5min":80.004000000000005,"15min":80.004000000000005},"last":11.048999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":15.539,"5min":18.071999999999999,"15min":20.584},"min":{"1min":10.071999999999999,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":21.744,"5min":87.866,"15min":87.866},"last":12.558999999999999},{"interface":"front","average":{"1min":15.82,"5min":19.992999999999999,"15min":22.632000000000001},"min":{"1min":8.4550000000000001,"5min":8.4550000000000001,"15min":8.4550000000000001},"max":{"1min":28.135000000000002,"5min":91.183000000000007,"15min":91.183000000000007},"last":28.201000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":12.398999999999999,"15min":12.625},"min":{"1min":6.7990000000000004,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":24.067,"5min":25.195,"15min":25.195},"last":33.393999999999998},{"interface":"front","average":{"1min":17.079999999999998,"5min":16.25,"15min":16.873999999999999},"min":{"1min":10.51,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":31.893000000000001,"5min":59.609000000000002,"15min":59.609000000000002},"last":23.263999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.930999999999999,"5min":15.021000000000001,"15min":15.755000000000001},"min":{"1min":7.9820000000000002,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":30.594000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":23.936},{"interface":"front","average":{"1min":18.491,"5min":19.263000000000002,"15min":20.593},"min":{"1min":13.439,"5min":9.4399999999999995,"15min":9.4399999999999995},"max":{"1min":33.707000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":20.021000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.425000000000001,"5min":18.123999999999999,"15min":18.806000000000001},"min":{"1min":10.709,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":29.001999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":35.192},{"interface":"front","average":{"1min":15.053000000000001,"5min":16.744,"15min":17.835000000000001},"min":{"1min":9.1760000000000002,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":20.460999999999999,"5min":42.531999999999996,"15min":42.531999999999996},"last":26.545999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":310709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":9.9199999999999999,"5min":9.6159999999999997,"15min":9.4120000000000008},"min":{"1min":4.9379999999999997,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":16.841000000000001,"5min":26.984999999999999,"15min":26.984999999999999},"last":7.2450000000000001},{"interface":"front","average":{"1min":10.007999999999999,"5min":10.661,"15min":10.592000000000001},"min":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"max":{"1min":15.428000000000001,"5min":25.934999999999999,"15min":25.934999999999999},"last":8.7100000000000009}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.516999999999999,"5min":12.371,"15min":11.771000000000001},"min":{"1min":7.0449999999999999,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":25.122,"5min":25.122,"15min":25.122},"last":9.4469999999999992},{"interface":"front","average":{"1min":12.935,"5min":13.680999999999999,"15min":13.994999999999999},"min":{"1min":8.3219999999999992,"5min":6.609,"15min":6.609},"max":{"1min":20.672999999999998,"5min":30.751999999999999,"15min":30.751999999999999},"last":14.019}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.712999999999999,"5min":13.198,"15min":12.378},"min":{"1min":7.5339999999999998,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":19.831,"5min":25.844000000000001,"15min":25.844000000000001},"last":21.844000000000001},{"interface":"front","average":{"1min":14.474,"5min":15.125999999999999,"15min":15.558999999999999},"min":{"1min":9.2080000000000002,"5min":9.1539999999999999,"15min":9.1539999999999999},"max":{"1min":23.298999999999999,"5min":25.428000000000001,"15min":25.428000000000001},"last":22.576000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":8.4220000000000006,"5min":10.686,"15min":11.071999999999999},"min":{"1min":6.1260000000000003,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":12.452999999999999,"5min":66.722999999999999,"15min":66.722999999999999},"last":7.9160000000000004},{"interface":"front","average":{"1min":15.635,"5min":16.170999999999999,"15min":16.100999999999999},"min":{"1min":8.1050000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"max":{"1min":27.145,"5min":69.346000000000004,"15min":69.346000000000004},"last":13.27}]},{"osd":5,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.201000000000001,"5min":18.181000000000001,"15min":19.129000000000001},"min":{"1min":9.4939999999999998,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":25.771999999999998,"5min":64.474999999999994,"15min":64.474999999999994},"last":12.021000000000001},{"interface":"front","average":{"1min":12.016,"5min":14.678000000000001,"15min":15.699999999999999},"min":{"1min":6.6440000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":32.332999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":12.587}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":18.664999999999999,"5min":18.544,"15min":18.699999999999999},"min":{"1min":12.724,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":38.712000000000003,"5min":38.712000000000003,"15min":38.712000000000003},"last":25.190000000000001},{"interface":"front","average":{"1min":14.061999999999999,"5min":13.598000000000001,"15min":13.882999999999999},"min":{"1min":8.8849999999999998,"5min":7.343,"15min":7.343},"max":{"1min":33.802999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":19.701000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.358000000000001,"5min":17.379999999999999,"15min":18.032},"min":{"1min":11.266,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":22.605,"5min":74.977999999999994,"15min":74.977999999999994},"last":24.344999999999999},{"interface":"front","average":{"1min":18.617999999999999,"5min":20.350000000000001,"15min":21.686},"min":{"1min":14.305999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":29.989000000000001,"5min":83.138000000000005,"15min":83.138000000000005},"last":20.901}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:29:09.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:29:09.995 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:09.995+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:10.067 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:10.067 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:10.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:10.097 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:10.096+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:10.198 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:10.198+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:10.241 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:10.241 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:10.242 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:10.242 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:10.242 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:10.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:10.335 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:10.334+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:10.364 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:29:10.422 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:10.422+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:10.462 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.463 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.30641207799999, 2023-10-30T06:29:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:10.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:10.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.474 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.476 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 295.393238215, 2023-10-30T06:29:10.477 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:10.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.478 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.479 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.479 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:10.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.479 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:10.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:10.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:10.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:10.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:10.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:10.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:10.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:10.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:10.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:10.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:10.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:10.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:10.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:10.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:10.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:10.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:10.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.489 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.490 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:10.490 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:10.490 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 294.73306024599998, 2023-10-30T06:29:10.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:10.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.491 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.491 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:10.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:10.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:10.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:10.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:10.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:10.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:10.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:10.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:10.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:10.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:10.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:10.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:10.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:10.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:10.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:10.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.502 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.502 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.502 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:10.503 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:10.503 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 294.65763703900001, 2023-10-30T06:29:10.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:10.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:10.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:10.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:10.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:10.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:10.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:10.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:10.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:10.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:10.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:10.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:10.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:10.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:10.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.513 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.513 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.514 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:10.514 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:10.514 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 294.38651390799998, 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.515 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.516 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:10.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:10.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:10.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:10.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:10.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:10.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:10.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:10.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:10.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:10.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:10.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:10.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:10.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:10.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:10.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.525 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:10.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:10.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.525 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.526 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.526 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:10.526 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 294.35623419799998, 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.527 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.528 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.529 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:10.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:10.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:10.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:10.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:10.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:10.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:10.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:10.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:10.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:10.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:10.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:10.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:10.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:10.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.537 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.537 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.538 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:10.538 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:10.538 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.982323114, 2023-10-30T06:29:10.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.539 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:10.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.540 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:10.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:10.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:10.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:10.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:10.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:10.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:10.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:10.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:10.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:10.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:10.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:10.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:10.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:10.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:10.549 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.550 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.550 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.550 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:10.551 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:10.551 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.97782441300001, 2023-10-30T06:29:10.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:10.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.552 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:10.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:10.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:10.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:10.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:10.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:10.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:10.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:10.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:10.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:10.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:10.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:10.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:10.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:10.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:10.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:10.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:10.562 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.562 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.562 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.563 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:10.563 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:10.563 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.68622047399998, 2023-10-30T06:29:10.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:10.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:10.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:10.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:10.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:10.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:10.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:10.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:10.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:10.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.574 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.68028315700002, 2023-10-30T06:29:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.576 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:10.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:10.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:10.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:10.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:10.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:10.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:10.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:10.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:10.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:10.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:10.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:10.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:10.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.585 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.67898414199999, 2023-10-30T06:29:10.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:10.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:10.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:10.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:10.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:10.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:10.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:10.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:10.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:10.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:10.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:10.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:10.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:10.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.597 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.57599533400003, 2023-10-30T06:29:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:10.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:10.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:10.608 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:10.609 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.304092067, 2023-10-30T06:29:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.610 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.29659387499999, 2023-10-30T06:29:10.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:10.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:10.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:10.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:10.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:10.634 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.29593267000001, 2023-10-30T06:29:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.28334683200001, 2023-10-30T06:29:10.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:10.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:10.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:10.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:10.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:10.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:10.657 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.658 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.27725461900002, 2023-10-30T06:29:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:10.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:10.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:10.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:10.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:10.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:10.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:10.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 293.276259278, 2023-10-30T06:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.681 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:10.682 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 285.953586707, 2023-10-30T06:29:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:10.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 285.92014522199997, 2023-10-30T06:29:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:10.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:10.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:10.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:10.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:10.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:10.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:10.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:10.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:10.704 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:10.705 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:10.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2023-10-30T06:29:10.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:10.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:29:10.709 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:29:10.710 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2023-10-30T06:29:10.365548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221768,"kb_used_data":5596,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752952,"statfs":{"total":773094113280,"available":772867022848,"internally_reserved":0,"allocated":5730304,"data_stored":3320111,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":66,"apply_latency_ms":66,"commit_latency_ns":66000000,"apply_latency_ns":66000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804540"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.737032+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:29:03.737032+0000","last_peered":"2023-10-30T06:29:03.737032+0000","last_clean":"2023-10-30T06:29:03.737032+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:29:03.737032+0000","last_undegraded":"2023-10-30T06:29:03.737032+0000","last_fullsized":"2023-10-30T06:29:03.737032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.911685+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:29:02.911685+0000","last_peered":"2023-10-30T06:29:02.911685+0000","last_clean":"2023-10-30T06:29:02.911685+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:29:02.911685+0000","last_undegraded":"2023-10-30T06:29:02.911685+0000","last_fullsized":"2023-10-30T06:29:02.911685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.568467+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:29:05.568467+0000","last_peered":"2023-10-30T06:29:05.568467+0000","last_clean":"2023-10-30T06:29:05.568467+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:29:05.568467+0000","last_undegraded":"2023-10-30T06:29:05.568467+0000","last_fullsized":"2023-10-30T06:29:05.568467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.558305+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:29:02.558305+0000","last_peered":"2023-10-30T06:29:02.558305+0000","last_clean":"2023-10-30T06:29:02.558305+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:29:02.558305+0000","last_undegraded":"2023-10-30T06:29:02.558305+0000","last_fullsized":"2023-10-30T06:29:02.558305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.732899+0000","last_change":"2023-10-30T06:28:34.904618+0000","last_active":"2023-10-30T06:29:03.732899+0000","last_peered":"2023-10-30T06:29:03.732899+0000","last_clean":"2023-10-30T06:29:03.732899+0000","last_became_active":"2023-10-30T06:28:34.898267+0000","last_became_peered":"2023-10-30T06:28:34.898267+0000","last_unstale":"2023-10-30T06:29:03.732899+0000","last_undegraded":"2023-10-30T06:29:03.732899+0000","last_fullsized":"2023-10-30T06:29:03.732899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:12.089156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.668969+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:29:02.668969+0000","last_peered":"2023-10-30T06:29:02.668969+0000","last_clean":"2023-10-30T06:29:02.668969+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:29:02.668969+0000","last_undegraded":"2023-10-30T06:29:02.668969+0000","last_fullsized":"2023-10-30T06:29:02.668969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.729529+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:29:03.729529+0000","last_peered":"2023-10-30T06:29:03.729529+0000","last_clean":"2023-10-30T06:29:03.729529+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:29:03.729529+0000","last_undegraded":"2023-10-30T06:29:03.729529+0000","last_fullsized":"2023-10-30T06:29:03.729529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":49,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-30T06:29:02.667265+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:29:02.667265+0000","last_peered":"2023-10-30T06:29:02.667265+0000","last_clean":"2023-10-30T06:29:02.667265+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:29:02.667265+0000","last_undegraded":"2023-10-30T06:29:02.667265+0000","last_fullsized":"2023-10-30T06:29:02.667265+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.563550+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:29:05.563550+0000","last_peered":"2023-10-30T06:29:05.563550+0000","last_clean":"2023-10-30T06:29:05.563550+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:29:05.563550+0000","last_undegraded":"2023-10-30T06:29:05.563550+0000","last_fullsized":"2023-10-30T06:29:05.563550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411391,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":719623,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.114000000000001,"5min":10.382999999999999,"15min":10.367000000000001},"min":{"1min":4.8570000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":41.07,"5min":41.07,"15min":41.07},"last":7.7409999999999997},{"interface":"front","average":{"1min":11.855,"5min":11.709,"15min":11.958},"min":{"1min":6.0679999999999996,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":44.597999999999999,"5min":46.798000000000002,"15min":46.798000000000002},"last":8.891}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.618,"5min":12.460000000000001,"15min":13.352},"min":{"1min":6.2060000000000004,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":23.099,"5min":52.552,"15min":52.552},"last":8.1029999999999998},{"interface":"front","average":{"1min":13.275,"5min":14.632,"15min":14.975},"min":{"1min":8.3219999999999992,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":25.076000000000001,"5min":55.585999999999999,"15min":55.585999999999999},"last":11.816000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":15.544,"5min":18.59,"15min":21.657},"min":{"1min":9.6660000000000004,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":23.303999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":15.771000000000001},{"interface":"front","average":{"1min":14.035,"5min":17.509,"15min":20.884},"min":{"1min":8.2300000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":31.236000000000001,"5min":124.075,"15min":124.075},"last":12.601000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":13.478999999999999,"5min":13.329000000000001,"15min":13.558999999999999},"min":{"1min":7.1319999999999997,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":32.243000000000002,"5min":51.649000000000001,"15min":51.649000000000001},"last":9.9900000000000002},{"interface":"front","average":{"1min":16.407,"5min":17.34,"15min":17.779},"min":{"1min":8.8810000000000002,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":25.788,"5min":44.973999999999997,"15min":44.973999999999997},"last":13.694000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":15.675000000000001,"5min":17.978000000000002,"15min":20.027000000000001},"min":{"1min":8.7349999999999994,"5min":7.351,"15min":7.351},"max":{"1min":28.606999999999999,"5min":123.754,"15min":123.754},"last":14.321},{"interface":"front","average":{"1min":16.48,"5min":18.271999999999998,"15min":21.477},"min":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"max":{"1min":35.601999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":10.672000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":17.298999999999999,"5min":19.186,"15min":20.699999999999999},"min":{"1min":9.3279999999999994,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":33.847000000000001,"5min":66.400999999999996,"15min":66.400999999999996},"last":20.018000000000001},{"interface":"front","average":{"1min":17.619,"5min":18.045999999999999,"15min":19.254000000000001},"min":{"1min":8.1929999999999996,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":43.159999999999997,"5min":53.149000000000001,"15min":53.149000000000001},"last":22.119}]},{"osd":7,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":17.652000000000001,"5min":19.731000000000002,"15min":20.919},"min":{"1min":11.337999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":27.016999999999999,"5min":51.375,"15min":51.375},"last":17.085999999999999},{"interface":"front","average":{"1min":18.718,"5min":20.405000000000001,"15min":21.198},"min":{"1min":10.598000000000001,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":38.033999999999999,"5min":69.991,"15min":69.991},"last":15.066000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476800,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":321783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.73,"5min":12.119,"15min":13.843},"min":{"1min":5.3339999999999996,"5min":4.7779999999999996,"15min":4.7779999999999996},"max":{"1min":26.062000000000001,"5min":94.343000000000004,"15min":94.343000000000004},"last":6.0519999999999996},{"interface":"front","average":{"1min":10.679,"5min":12.59,"15min":14.699},"min":{"1min":5.0049999999999999,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":22.806000000000001,"5min":122.054,"15min":122.054},"last":7.1070000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.141,"5min":10.465999999999999,"15min":10.699},"min":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"max":{"1min":21.786000000000001,"5min":21.786000000000001,"15min":21.786000000000001},"last":19.745999999999999},{"interface":"front","average":{"1min":15.263999999999999,"5min":21.969000000000001,"15min":29.899000000000001},"min":{"1min":5.9989999999999997,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":36.935000000000002,"5min":387.15600000000001,"15min":387.15600000000001},"last":20.484999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.308999999999999,"5min":15.284000000000001,"15min":18.398},"min":{"1min":6.6799999999999997,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":16.988,"5min":156.90299999999999,"15min":156.90299999999999},"last":22.667999999999999},{"interface":"front","average":{"1min":15.432,"5min":17.199000000000002,"15min":19.454999999999998},"min":{"1min":11.369999999999999,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":22.655000000000001,"5min":143.578,"15min":143.578},"last":23.745000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":18.914999999999999,"15min":20.946999999999999},"min":{"1min":7.7999999999999998,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":27.901,"5min":141.18799999999999,"15min":141.18799999999999},"last":21.300000000000001},{"interface":"front","average":{"1min":18.728000000000002,"5min":21.913,"15min":24.251000000000001},"min":{"1min":9.1099999999999994,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":35.975999999999999,"5min":161.32900000000001,"15min":161.32900000000001},"last":29.498999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.367000000000001,"5min":13.390000000000001,"15min":14.493},"min":{"1min":7.835,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":22.731000000000002,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.5069999999999997},{"interface":"front","average":{"1min":15.497,"5min":15.002000000000001,"15min":15.468},"min":{"1min":7.6379999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"max":{"1min":24.989999999999998,"5min":58.613999999999997,"15min":58.613999999999997},"last":25.407}]},{"osd":5,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":13.66,"5min":15.308,"15min":17.079999999999998},"min":{"1min":8.4019999999999992,"5min":7.383,"15min":7.383},"max":{"1min":24.629999999999999,"5min":71.757000000000005,"15min":71.757000000000005},"last":10.41},{"interface":"front","average":{"1min":18.666,"5min":19.806999999999999,"15min":21.109999999999999},"min":{"1min":9.532,"5min":9.532,"15min":9.532},"max":{"1min":29.719999999999999,"5min":80.119,"15min":80.119},"last":24.709}]},{"osd":7,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":17.417000000000002,"15min":18.966000000000001},"min":{"1min":9.9329999999999998,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":23.734999999999999,"5min":81.247,"15min":81.247},"last":11.686999999999999},{"interface":"front","average":{"1min":16.643999999999998,"5min":17.164999999999999,"15min":18.446999999999999},"min":{"1min":8.6310000000000002,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":31.199000000000002,"5min":70.989000000000004,"15min":70.989000000000004},"last":27.321999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476800,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":10.869999999999999,"15min":10.996},"min":{"1min":4.9450000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":48.792000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":12.890000000000001},{"interface":"front","average":{"1min":11.84,"5min":10.802,"15min":10.439},"min":{"1min":7.0129999999999999,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":28.616,"5min":28.616,"15min":28.616},"last":6.915}]},{"osd":1,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":13.914999999999999,"5min":19.594000000000001,"15min":24.416},"min":{"1min":7.46,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":33.198,"5min":165.76400000000001,"15min":165.76400000000001},"last":15.276},{"interface":"front","average":{"1min":13.18,"5min":19.954999999999998,"15min":25.370999999999999},"min":{"1min":7.8289999999999997,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":21.282,"5min":190.34200000000001,"15min":190.34200000000001},"last":12.006}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":16.754999999999999,"15min":17.870000000000001},"min":{"1min":8.548,"5min":7.96,"15min":7.96},"max":{"1min":34.350000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":13.701000000000001},{"interface":"front","average":{"1min":15.375999999999999,"5min":15.984,"15min":16.794},"min":{"1min":8.109,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":47.140999999999998,"5min":47.140999999999998,"15min":47.140999999999998},"last":17.533000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.747,"5min":18.550999999999998,"15min":19.93},"min":{"1min":11.375,"5min":9.8979999999999997,"15min":9.8979999999999997},"max":{"1min":44.091000000000001,"5min":44.091000000000001,"15min":44.091000000000001},"last":16.076000000000001},{"interface":"front","average":{"1min":12.366,"5min":11.036,"15min":11.420999999999999},"min":{"1min":5.9400000000000004,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":45.231000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.2550000000000008}]},{"osd":5,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.690999999999999,"5min":16.504999999999999,"15min":18.606999999999999},"min":{"1min":11.563000000000001,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":49.784999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":19.428000000000001},{"interface":"front","average":{"1min":13.282999999999999,"5min":17.286999999999999,"15min":19.628},"min":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":28.132999999999999,"5min":58.576000000000001,"15min":58.576000000000001},"last":8.8559999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":15.202,"5min":22.263999999999999,"15min":26.846},"min":{"1min":8.4260000000000002,"5min":8.4260000000000002,"15min":8.4260000000000002},"max":{"1min":29.905000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":9.9979999999999993},{"interface":"front","average":{"1min":16.574000000000002,"5min":14.781000000000001,"15min":15.256},"min":{"1min":9.5060000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":34.969000000000001,"5min":40.128999999999998,"15min":40.128999999999998},"last":10.561}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.837,"5min":18.343,"15min":18.105},"min":{"1min":12.76,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":50.722000000000001,"5min":50.722000000000001,"15min":50.722000000000001},"last":21.396999999999998},{"interface":"front","average":{"1min":18.157,"5min":23.681000000000001,"15min":27.637},"min":{"1min":11.978999999999999,"5min":10.025,"15min":10.025},"max":{"1min":29.071999999999999,"5min":170.232,"15min":170.232},"last":19.818000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":9.9510000000000005,"5min":17.408999999999999,"15min":24.341999999999999},"min":{"1min":5.4710000000000001,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":27.117000000000001,"5min":258.66199999999998,"15min":258.66199999999998},"last":15.102},{"interface":"front","average":{"1min":10.667,"5min":15.890000000000001,"15min":21.158000000000001},"min":{"1min":5.3499999999999996,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":28.109999999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":11.819000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.951000000000001,"5min":14.247999999999999,"15min":15.622999999999999},"min":{"1min":7.468,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":27.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":13.709},{"interface":"front","average":{"1min":12.977,"5min":13.169,"15min":14.282},"min":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":6.6070000000000002},"max":{"1min":21.780999999999999,"5min":37.390999999999998,"15min":37.390999999999998},"last":19.134}]},{"osd":3,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.968999999999999,"5min":14.577,"15min":17.742999999999999},"min":{"1min":6.8899999999999997,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":23.518999999999998,"5min":183.38900000000001,"15min":183.38900000000001},"last":6.8019999999999996},{"interface":"front","average":{"1min":16.148,"5min":18.513000000000002,"15min":21.548999999999999},"min":{"1min":7.6399999999999997,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.055,"5min":189.374,"15min":189.374},"last":15.715}]},{"osd":4,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.247999999999999,"5min":17.015000000000001,"15min":20.311},"min":{"1min":8.3019999999999996,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":16.420999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":9.2040000000000006},{"interface":"front","average":{"1min":16.623000000000001,"5min":16.381,"15min":18.870000000000001},"min":{"1min":8.9849999999999994,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":36.784999999999997,"5min":167.67599999999999,"15min":167.67599999999999},"last":18.524000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":18.126999999999999,"5min":21.486000000000001,"15min":25.535},"min":{"1min":10.58,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":24.699000000000002,"5min":215.80500000000001,"15min":215.80500000000001},"last":16.844999999999999},{"interface":"front","average":{"1min":17.141999999999999,"5min":20.536000000000001,"15min":25.065999999999999},"min":{"1min":8.8520000000000003,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":33.207999999999998,"5min":187.863,"15min":187.863},"last":13.087}]},{"osd":6,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.503,"5min":20.372,"15min":24.965},"min":{"1min":11.701000000000001,"5min":10.32,"15min":10.32},"max":{"1min":28.986999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":7.7969999999999997},{"interface":"front","average":{"1min":20.504999999999999,"5min":23.777000000000001,"15min":28.158000000000001},"min":{"1min":12.313000000000001,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":33.987000000000002,"5min":275.92200000000003,"15min":275.92200000000003},"last":20.614999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":19.542999999999999,"5min":21.734999999999999,"15min":25.396000000000001},"min":{"1min":9.4580000000000002,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":35.898000000000003,"5min":170.43600000000001,"15min":170.43600000000001},"last":17.850999999999999},{"interface":"front","average":{"1min":18.407,"5min":19.861999999999998,"15min":22.312000000000001},"min":{"1min":8.5280000000000005,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.297000000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":10.759}]}]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":9.4700000000000006,"5min":10.786,"15min":10.725},"min":{"1min":4.9829999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":25.042999999999999,"5min":56.704000000000001,"15min":56.704000000000001},"last":10.776},{"interface":"front","average":{"1min":10.782999999999999,"5min":11.834,"15min":11.882999999999999},"min":{"1min":5.8559999999999999,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":26.018999999999998,"5min":57.805,"15min":57.805},"last":10.196999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":10.135,"5min":13.997,"15min":18.033000000000001},"min":{"1min":6.5469999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":17.658000000000001,"5min":127.95399999999999,"15min":127.95399999999999},"last":8.7260000000000009},{"interface":"front","average":{"1min":13.305,"5min":13.374000000000001,"15min":14.372},"min":{"1min":7.4390000000000001,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":35.310000000000002,"5min":35.834000000000003,"15min":35.834000000000003},"last":12.269}]},{"osd":3,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.784000000000001,"5min":16.899000000000001,"15min":20.733000000000001},"min":{"1min":8.6069999999999993,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":18.431000000000001,"5min":99.790999999999997,"15min":99.790999999999997},"last":20.157},{"interface":"front","average":{"1min":15.534000000000001,"5min":15.677,"15min":16.369},"min":{"1min":10.504,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":43.860999999999997,"5min":43.860999999999997,"15min":43.860999999999997},"last":21.922999999999998}]},{"osd":4,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":14.250999999999999,"5min":14.385999999999999,"15min":15.895},"min":{"1min":8.2260000000000009,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":51.802,"5min":51.802,"15min":51.802},"last":7.1559999999999997},{"interface":"front","average":{"1min":18.021000000000001,"5min":19.768000000000001,"15min":23.504999999999999},"min":{"1min":10.582000000000001,"5min":7.6689999999999996,"15min":7.6689999999999996},"max":{"1min":40.000999999999998,"5min":103.705,"15min":103.705},"last":14.912000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.994,"5min":15.896000000000001,"15min":17.991},"min":{"1min":9.1519999999999992,"5min":7.8330000000000002,"15min":7.8330000000000002},"max":{"1min":17.576000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":12.898999999999999},{"interface":"front","average":{"1min":19.388000000000002,"5min":21.844999999999999,"15min":26.18},"min":{"1min":9.3119999999999994,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":40.598999999999997,"5min":98.707999999999998,"15min":98.707999999999998},"last":18.882000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.209,"5min":19.010999999999999,"15min":20.006},"min":{"1min":11.734,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":51.018999999999998,"5min":55.631,"15min":55.631},"last":16.119},{"interface":"front","average":{"1min":16.334,"5min":19.213000000000001,"15min":21.271000000000001},"min":{"1min":9.907,"5min":9.7159999999999993,"15min":9.7159999999999993},"max":{"1min":48.860999999999997,"5min":54.656999999999996,"15min":54.656999999999996},"last":17.809000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.692,"5min":19.841000000000001,"15min":21.363},"min":{"1min":12.294,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":53.314,"5min":53.314,"15min":53.314},"last":22.963999999999999},{"interface":"front","average":{"1min":18.108000000000001,"5min":20.082000000000001,"15min":22.216000000000001},"min":{"1min":10.589,"5min":10.589,"15min":10.589},"max":{"1min":37.427999999999997,"5min":45.130000000000003,"15min":45.130000000000003},"last":15.539999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":310709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.51,"5min":12.632999999999999,"15min":15.106999999999999},"min":{"1min":4.944,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":40.948,"5min":131.477,"15min":131.477},"last":16.724},{"interface":"front","average":{"1min":11.792999999999999,"5min":13.177,"15min":15.381},"min":{"1min":6.4649999999999999,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":40.159999999999997,"5min":129.386,"15min":129.386},"last":7.0339999999999998}]},{"osd":1,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":10.955,"5min":10.936999999999999,"15min":10.975},"min":{"1min":6.8940000000000001,"5min":6.3390000000000004,"15min":6.3390000000000004},"max":{"1min":18.311,"5min":25.774000000000001,"15min":25.774000000000001},"last":10.638},{"interface":"front","average":{"1min":15.336,"5min":17.414999999999999,"15min":18.797000000000001},"min":{"1min":12.584,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":19.048999999999999,"5min":91.295000000000002,"15min":91.295000000000002},"last":17.68}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.068,"5min":14.746,"15min":17.061},"min":{"1min":6.5110000000000001,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":21.138999999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":12.663},{"interface":"front","average":{"1min":12.34,"5min":14.003,"15min":14.423999999999999},"min":{"1min":7.3540000000000001,"5min":5.992,"15min":5.992},"max":{"1min":22.401,"5min":50.450000000000003,"15min":50.450000000000003},"last":18.460999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":14.092000000000001,"5min":16.588999999999999,"15min":19.309000000000001},"min":{"1min":7.2750000000000004,"5min":6.758,"15min":6.758},"max":{"1min":22.478999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":8.2750000000000004},{"interface":"front","average":{"1min":16.396999999999998,"5min":17.763999999999999,"15min":20.045999999999999},"min":{"1min":11.826000000000001,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":22.414999999999999,"5min":124.684,"15min":124.684},"last":15.577}]},{"osd":4,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":14.701000000000001,"5min":13.255000000000001,"15min":13.677},"min":{"1min":10.32,"5min":6.6050000000000004,"15min":6.6050000000000004},"max":{"1min":21.858000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":13.587},{"interface":"front","average":{"1min":17.369,"5min":16.047000000000001,"15min":15.75},"min":{"1min":10.175000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":28.701000000000001,"5min":28.701000000000001,"15min":28.701000000000001},"last":20.657}]},{"osd":5,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":11.826000000000001,"5min":16.309999999999999,"15min":19.253},"min":{"1min":6.9550000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":20.228000000000002,"5min":125.792,"15min":125.792},"last":11.827999999999999},{"interface":"front","average":{"1min":16.962,"5min":16.856000000000002,"15min":16.956},"min":{"1min":12.593999999999999,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":28.417999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":19.93}]},{"osd":6,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.372,"5min":22.959,"15min":29.097999999999999},"min":{"1min":13.268000000000001,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":30.768999999999998,"5min":327.29599999999999,"15min":327.29599999999999},"last":21.893999999999998},{"interface":"front","average":{"1min":13.772,"5min":22.376000000000001,"15min":30.071000000000002},"min":{"1min":8.5779999999999994,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":25.986999999999998,"5min":323.82400000000001,"15min":323.82400000000001},"last":9.7859999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":708549,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":8.4239999999999995,"5min":11.695,"15min":13.956},"min":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":16.850000000000001,"5min":68.513999999999996,"15min":68.513999999999996},"last":8.5830000000000002},{"interface":"front","average":{"1min":9.6069999999999993,"5min":12.859999999999999,"15min":15.647},"min":{"1min":6.8209999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":14.297000000000001,"5min":88.816000000000003,"15min":88.816000000000003},"last":9.7910000000000004}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":12.551,"5min":15.057,"15min":16.774999999999999},"min":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":21.32,"5min":87.515000000000001,"15min":87.515000000000001},"last":32.015999999999998},{"interface":"front","average":{"1min":13.327,"5min":15.451000000000001,"15min":17.158999999999999},"min":{"1min":8.8840000000000003,"5min":6.21,"15min":6.21},"max":{"1min":22.943000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":36.521000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":14.614000000000001,"5min":16.452000000000002,"15min":17.725999999999999},"min":{"1min":6.5259999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":27.128,"5min":113.943,"15min":113.943},"last":25.742000000000001},{"interface":"front","average":{"1min":10.391,"5min":12.885999999999999,"15min":14.608000000000001},"min":{"1min":6.2889999999999997,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.457999999999998,"5min":80.004000000000005,"15min":80.004000000000005},"last":11.048999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":15.539,"5min":18.071999999999999,"15min":20.584},"min":{"1min":10.071999999999999,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":21.744,"5min":87.866,"15min":87.866},"last":12.558999999999999},{"interface":"front","average":{"1min":15.82,"5min":19.992999999999999,"15min":22.632000000000001},"min":{"1min":8.4550000000000001,"5min":8.4550000000000001,"15min":8.4550000000000001},"max":{"1min":28.135000000000002,"5min":91.183000000000007,"15min":91.183000000000007},"last":28.201000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":12.398999999999999,"15min":12.625},"min":{"1min":6.7990000000000004,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":24.067,"5min":25.195,"15min":25.195},"last":33.393999999999998},{"interface":"front","average":{"1min":17.079999999999998,"5min":16.25,"15min":16.873999999999999},"min":{"1min":10.51,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":31.893000000000001,"5min":59.609000000000002,"15min":59.609000000000002},"last":23.263999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.930999999999999,"5min":15.021000000000001,"15min":15.755000000000001},"min":{"1min":7.9820000000000002,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":30.594000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":23.936},{"interface":"front","average":{"1min":18.491,"5min":19.263000000000002,"15min":20.593},"min":{"1min":13.439,"5min":9.4399999999999995,"15min":9.4399999999999995},"max":{"1min":33.707000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":20.021000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.425000000000001,"5min":18.123999999999999,"15min":18.806000000000001},"min":{"1min":10.709,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":29.001999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":35.192},{"interface":"front","average":{"1min":15.053000000000001,"5min":16.744,"15min":17.835000000000001},"min":{"1min":9.1760000000000002,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":20.460999999999999,"5min":42.531999999999996,"15min":42.531999999999996},"last":26.545999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":310709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":9.9199999999999999,"5min":9.6159999999999997,"15min":9.4120000000000008},"min":{"1min":4.9379999999999997,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":16.841000000000001,"5min":26.984999999999999,"15min":26.984999999999999},"last":7.2450000000000001},{"interface":"front","average":{"1min":10.007999999999999,"5min":10.661,"15min":10.592000000000001},"min":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"max":{"1min":15.428000000000001,"5min":25.934999999999999,"15min":25.934999999999999},"last":8.7100000000000009}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.516999999999999,"5min":12.371,"15min":11.771000000000001},"min":{"1min":7.0449999999999999,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":25.122,"5min":25.122,"15min":25.122},"last":9.4469999999999992},{"interface":"front","average":{"1min":12.935,"5min":13.680999999999999,"15min":13.994999999999999},"min":{"1min":8.3219999999999992,"5min":6.609,"15min":6.609},"max":{"1min":20.672999999999998,"5min":30.751999999999999,"15min":30.751999999999999},"last":14.019}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.712999999999999,"5min":13.198,"15min":12.378},"min":{"1min":7.5339999999999998,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":19.831,"5min":25.844000000000001,"15min":25.844000000000001},"last":21.844000000000001},{"interface":"front","average":{"1min":14.474,"5min":15.125999999999999,"15min":15.558999999999999},"min":{"1min":9.2080000000000002,"5min":9.1539999999999999,"15min":9.1539999999999999},"max":{"1min":23.298999999999999,"5min":25.428000000000001,"15min":25.428000000000001},"last":22.576000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":8.4220000000000006,"5min":10.686,"15min":11.071999999999999},"min":{"1min":6.1260000000000003,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":12.452999999999999,"5min":66.722999999999999,"15min":66.722999999999999},"last":7.9160000000000004},{"interface":"front","average":{"1min":15.635,"5min":16.170999999999999,"15min":16.100999999999999},"min":{"1min":8.1050000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"max":{"1min":27.145,"5min":69.346000000000004,"15min":69.346000000000004},"last":13.27}]},{"osd":5,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.201000000000001,"5min":18.181000000000001,"15min":19.129000000000001},"min":{"1min":9.4939999999999998,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":25.771999999999998,"5min":64.474999999999994,"15min":64.474999999999994},"last":12.021000000000001},{"interface":"front","average":{"1min":12.016,"5min":14.678000000000001,"15min":15.699999999999999},"min":{"1min":6.6440000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":32.332999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":12.587}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":18.664999999999999,"5min":18.544,"15min":18.699999999999999},"min":{"1min":12.724,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":38.712000000000003,"5min":38.712000000000003,"15min":38.712000000000003},"last":25.190000000000001},{"interface":"front","average":{"1min":14.061999999999999,"5min":13.598000000000001,"15min":13.882999999999999},"min":{"1min":8.8849999999999998,"5min":7.343,"15min":7.343},"max":{"1min":33.802999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":19.701000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.358000000000001,"5min":17.379999999999999,"15min":18.032},"min":{"1min":11.266,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":22.605,"5min":74.977999999999994,"15min":74.977999999999994},"last":24.344999999999999},{"interface":"front","average":{"1min":18.617999999999999,"5min":20.350000000000001,"15min":21.686},"min":{"1min":14.305999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":29.989000000000001,"5min":83.138000000000005,"15min":83.138000000000005},"last":20.901}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:29:10.712 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-10-30T06:29:10.713 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:10.706+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:10.713 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:10.713 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:29:10.714 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:29:10.807 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:10.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:10.876 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:10.876 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:10.877 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:10.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:10.926 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:10.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:11.009 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:11.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:11.052 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:11.053 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:11.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:11.146 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:11.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:11.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:11.224 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:11.223+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:11.227 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:11.227 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:11.228 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:11.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:11.320 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:11.320+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:11.408 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:11.409 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:11.414 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:11.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:11.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:11.453 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:11.454 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.472 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:29:11.473 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 81, 2023-10-30T06:29:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:29:11.473 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:29:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:29:09.988452+0000", 2023-10-30T06:29:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:29:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:28:33.068330+0000", 2023-10-30T06:29:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:29:11.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2023-10-30T06:29:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:29:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:29:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:29:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:29:11.475 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:29:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:29:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:29:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:29:11.476 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:29:11.477 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:29:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:29:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:29:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:29:11.478 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:29:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:29:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:29:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:29:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:29:11.479 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:29:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:29:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:29:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:29:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:29:11.480 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:29:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:29:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:29:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:29:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:29:11.481 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:29:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:29:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:29:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:29:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:29:11.482 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:29:11.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:29:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:29:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:29:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:29:11.484 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:29:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:29:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:29:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:29:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:29:11.485 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:29:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:29:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:29:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:29:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:29:11.486 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:29:11.487 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:29:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:29:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:29:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:29:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:29:11.488 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:29:11.489 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:29:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:29:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:29:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:29:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:29:11.490 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:29:11.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:29:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:29:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:29:11.492 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:29:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:29:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:29:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:29:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:29:11.493 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:29:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:29:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:29:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:29:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:29:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:29:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:29:11.495 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:29:11.496 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:29:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:29:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:29:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:29:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:29:11.497 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:29:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:29:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:29:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:29:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:29:11.498 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:29:11.499 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:29:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:29:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:29:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:29:11.500 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:29:11.501 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:29:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:29:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:29:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:29:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:29:11.502 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:29:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:29:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:29:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:29:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:29:11.503 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:29:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:29:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:29:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:29:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:29:11.504 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:29:11.505 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:29:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:29:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:29:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:29:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:29:11.506 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:29:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:29:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:29:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:29:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:29:11.507 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 2.630000114440918, 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 2.630000114440918, 2023-10-30T06:29:11.508 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:29:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3, 2023-10-30T06:29:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3, 2023-10-30T06:29:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.875, 2023-10-30T06:29:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 1, 2023-10-30T06:29:11.509 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2023-10-30T06:29:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:29:11.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.511 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:29:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:29:11.512 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:29:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.514 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.515 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:29:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:29:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.520 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:29:11.521 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:29:11.522 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:29:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:29:11.523 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:29:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:29:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.525 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.526 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:29:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 33, 2023-10-30T06:29:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.527 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:29:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.530 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:29:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.531 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.532 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:29:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.536 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:29:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:29:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.537 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:29:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:29:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:29:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:29:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.538 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.539 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:29:11.540 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:29:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 26, 2023-10-30T06:29:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.541 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.542 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:29:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.543 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:29:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.544 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:29:11.546 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.548 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.549 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:29:11.550 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:29:11.551 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:29:11.552 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:29:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:29:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.554 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:29:11.555 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 69, 2023-10-30T06:29:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:29:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:29:11.560 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:29:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:29:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.565 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:29:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:29:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.566 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:29:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:29:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:29:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:29:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.567 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:29:11.568 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:29:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.569 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:29:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 21, 2023-10-30T06:29:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.570 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:29:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.572 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:29:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.573 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:29:11.574 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:29:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:29:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.579 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:29:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.580 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:29:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:29:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.581 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.582 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:29:11.583 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:29:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.584 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 69, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.585 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:29:11.586 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:29:11.587 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:29:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:29:11.591 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:29:11.592 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:29:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:29:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:29:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:29:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:29:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:29:11.596 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:29:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:29:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.598 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.599 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:29:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:29:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.604 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:29:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:29:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.607 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.608 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:29:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.609 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:29:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:29:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:29:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:29:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:29:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:29:11.611 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:29:11.613 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:29:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:29:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:29:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:29:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:29:11.614 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:29:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:29:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:29:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:29:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.616 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:29:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:29:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.618 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:29:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.619 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:29:11.620 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:29:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.622 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:29:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:29:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:29:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:29:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.624 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:29:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:29:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:29:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:29:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:29:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:29:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:29:11.626 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.627 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:29:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:29:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.628 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:29:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.630 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:29:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:29:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.632 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.633 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:29:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.634 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:29:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.635 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.636 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:29:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:29:11.637 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.638 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.639 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:29:11.640 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:29:11.641 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:29:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:29:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:29:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:29:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:29:11.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:29:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:29:11.643 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.644 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:29:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.3", 2023-10-30T06:29:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:29:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:11.645 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 3, 2023-10-30T06:29:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 7 2023-10-30T06:29:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.646 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:29:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:29:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:29:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:29:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:29:11.647 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:29:11.648 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:29:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:29:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:29:11.649 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:29:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:29:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:29:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:29:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:29:11.650 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:29:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:29:11.653 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:29:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:29:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:29:11.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:11.654 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:11.656 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:11.657 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:11.658 INFO:tasks.thrashosds.thrasher:j is {'epoch': 81, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:29:09.988452+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:28:33.068330+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 33, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [1, 2]}], 'pg_upmap_items': [{'pgid': '2.3', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:29:11.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:29:11.659 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:11.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:11.660 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.660 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:11.661 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.661 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:11.655+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:11.722 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:11.722 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.756 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:11.756+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:11.801 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 214.826874659, 2023-10-30T06:29:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 34.806382335000002, 2023-10-30T06:29:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:11.830 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:11.830 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:11.831 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:11.877 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:11.875+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:11.910 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:11.911 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:11.958 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:11.956+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:11.965 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:11.965 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:11.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:11.995 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:11.995 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:12.077 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:12.077 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:12.078 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:12.076+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:12.126 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:12.127 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:12.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:12.176 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:12.176+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:12.260 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:12.260+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:12.266 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:29:12.267 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:29:12.279 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2023-10-30T06:29:11.307877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221916,"kb_used_data":5740,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752804,"statfs":{"total":773094113280,"available":772866871296,"internally_reserved":0,"allocated":5877760,"data_stored":3386555,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":68,"apply_latency_ms":68,"commit_latency_ns":68000000,"apply_latency_ns":68000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.580975"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.737032+0000","last_change":"2023-10-30T06:24:20.685516+0000","last_active":"2023-10-30T06:29:03.737032+0000","last_peered":"2023-10-30T06:29:03.737032+0000","last_clean":"2023-10-30T06:29:03.737032+0000","last_became_active":"2023-10-30T06:24:20.680410+0000","last_became_peered":"2023-10-30T06:24:20.680410+0000","last_unstale":"2023-10-30T06:29:03.737032+0000","last_undegraded":"2023-10-30T06:29:03.737032+0000","last_fullsized":"2023-10-30T06:29:03.737032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:19:42.602520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-30T06:29:06.737204+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:29:06.737204+0000","last_peered":"2023-10-30T06:29:06.737204+0000","last_clean":"2023-10-30T06:29:06.737204+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:29:06.737204+0000","last_undegraded":"2023-10-30T06:29:06.737204+0000","last_fullsized":"2023-10-30T06:29:06.737204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.568467+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:29:05.568467+0000","last_peered":"2023-10-30T06:29:05.568467+0000","last_clean":"2023-10-30T06:29:05.568467+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:29:05.568467+0000","last_undegraded":"2023-10-30T06:29:05.568467+0000","last_fullsized":"2023-10-30T06:29:05.568467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-30T06:29:08.435299+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:29:08.435299+0000","last_peered":"2023-10-30T06:29:08.435299+0000","last_clean":"2023-10-30T06:29:08.435299+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:29:08.435299+0000","last_undegraded":"2023-10-30T06:29:08.435299+0000","last_fullsized":"2023-10-30T06:29:08.435299+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.732899+0000","last_change":"2023-10-30T06:28:34.904618+0000","last_active":"2023-10-30T06:29:03.732899+0000","last_peered":"2023-10-30T06:29:03.732899+0000","last_clean":"2023-10-30T06:29:03.732899+0000","last_became_active":"2023-10-30T06:28:34.898267+0000","last_became_peered":"2023-10-30T06:28:34.898267+0000","last_unstale":"2023-10-30T06:29:03.732899+0000","last_undegraded":"2023-10-30T06:29:03.732899+0000","last_fullsized":"2023-10-30T06:29:03.732899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:06:12.089156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-30T06:29:06.783900+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:29:06.783900+0000","last_peered":"2023-10-30T06:29:06.783900+0000","last_clean":"2023-10-30T06:29:06.783900+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:29:06.783900+0000","last_undegraded":"2023-10-30T06:29:06.783900+0000","last_fullsized":"2023-10-30T06:29:06.783900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-30T06:29:03.729529+0000","last_change":"2023-10-30T06:24:20.672771+0000","last_active":"2023-10-30T06:29:03.729529+0000","last_peered":"2023-10-30T06:29:03.729529+0000","last_clean":"2023-10-30T06:29:03.729529+0000","last_became_active":"2023-10-30T06:24:20.658224+0000","last_became_peered":"2023-10-30T06:24:20.658224+0000","last_unstale":"2023-10-30T06:29:03.729529+0000","last_undegraded":"2023-10-30T06:29:03.729529+0000","last_fullsized":"2023-10-30T06:29:03.729529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:03:06.921547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-10-30T06:29:06.782196+0000","last_change":"2023-10-30T06:25:24.812868+0000","last_active":"2023-10-30T06:29:06.782196+0000","last_peered":"2023-10-30T06:29:06.782196+0000","last_clean":"2023-10-30T06:29:06.782196+0000","last_became_active":"2023-10-30T06:25:24.806809+0000","last_became_peered":"2023-10-30T06:25:24.806809+0000","last_unstale":"2023-10-30T06:29:06.782196+0000","last_undegraded":"2023-10-30T06:29:06.782196+0000","last_fullsized":"2023-10-30T06:29:06.782196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T11:42:56.200125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-30T06:29:05.563550+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:29:05.563550+0000","last_peered":"2023-10-30T06:29:05.563550+0000","last_clean":"2023-10-30T06:29:05.563550+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:29:05.563550+0000","last_undegraded":"2023-10-30T06:29:05.563550+0000","last_fullsized":"2023-10-30T06:29:05.563550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411391,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":719623,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.114000000000001,"5min":10.382999999999999,"15min":10.367000000000001},"min":{"1min":4.8570000000000002,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":41.07,"5min":41.07,"15min":41.07},"last":7.7409999999999997},{"interface":"front","average":{"1min":11.855,"5min":11.709,"15min":11.958},"min":{"1min":6.0679999999999996,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":44.597999999999999,"5min":46.798000000000002,"15min":46.798000000000002},"last":8.891}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.618,"5min":12.460000000000001,"15min":13.352},"min":{"1min":6.2060000000000004,"5min":5.5209999999999999,"15min":5.5209999999999999},"max":{"1min":23.099,"5min":52.552,"15min":52.552},"last":8.1029999999999998},{"interface":"front","average":{"1min":13.275,"5min":14.632,"15min":14.975},"min":{"1min":8.3219999999999992,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":25.076000000000001,"5min":55.585999999999999,"15min":55.585999999999999},"last":11.816000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":15.544,"5min":18.59,"15min":21.657},"min":{"1min":9.6660000000000004,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":23.303999999999998,"5min":118.45699999999999,"15min":118.45699999999999},"last":15.771000000000001},{"interface":"front","average":{"1min":14.035,"5min":17.509,"15min":20.884},"min":{"1min":8.2300000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":31.236000000000001,"5min":124.075,"15min":124.075},"last":12.601000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":13.478999999999999,"5min":13.329000000000001,"15min":13.558999999999999},"min":{"1min":7.1319999999999997,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":32.243000000000002,"5min":51.649000000000001,"15min":51.649000000000001},"last":9.9900000000000002},{"interface":"front","average":{"1min":16.407,"5min":17.34,"15min":17.779},"min":{"1min":8.8810000000000002,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":25.788,"5min":44.973999999999997,"15min":44.973999999999997},"last":13.694000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":15.675000000000001,"5min":17.978000000000002,"15min":20.027000000000001},"min":{"1min":8.7349999999999994,"5min":7.351,"15min":7.351},"max":{"1min":28.606999999999999,"5min":123.754,"15min":123.754},"last":14.321},{"interface":"front","average":{"1min":16.48,"5min":18.271999999999998,"15min":21.477},"min":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"max":{"1min":35.601999999999997,"5min":122.72499999999999,"15min":122.72499999999999},"last":10.672000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:20 2023","interfaces":[{"interface":"back","average":{"1min":17.298999999999999,"5min":19.186,"15min":20.699999999999999},"min":{"1min":9.3279999999999994,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":33.847000000000001,"5min":66.400999999999996,"15min":66.400999999999996},"last":20.018000000000001},{"interface":"front","average":{"1min":17.619,"5min":18.045999999999999,"15min":19.254000000000001},"min":{"1min":8.1929999999999996,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":43.159999999999997,"5min":53.149000000000001,"15min":53.149000000000001},"last":22.119}]},{"osd":7,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":17.652000000000001,"5min":19.731000000000002,"15min":20.919},"min":{"1min":11.337999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":27.016999999999999,"5min":51.375,"15min":51.375},"last":17.085999999999999},{"interface":"front","average":{"1min":18.718,"5min":20.405000000000001,"15min":21.198},"min":{"1min":10.598000000000001,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":38.033999999999999,"5min":69.991,"15min":69.991},"last":15.066000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476800,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":321783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.73,"5min":12.119,"15min":13.843},"min":{"1min":5.3339999999999996,"5min":4.7779999999999996,"15min":4.7779999999999996},"max":{"1min":26.062000000000001,"5min":94.343000000000004,"15min":94.343000000000004},"last":6.0519999999999996},{"interface":"front","average":{"1min":10.679,"5min":12.59,"15min":14.699},"min":{"1min":5.0049999999999999,"5min":4.7690000000000001,"15min":4.7690000000000001},"max":{"1min":22.806000000000001,"5min":122.054,"15min":122.054},"last":7.1070000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.141,"5min":10.465999999999999,"15min":10.699},"min":{"1min":5.5590000000000002,"5min":5.5590000000000002,"15min":5.5590000000000002},"max":{"1min":21.786000000000001,"5min":21.786000000000001,"15min":21.786000000000001},"last":19.745999999999999},{"interface":"front","average":{"1min":15.263999999999999,"5min":21.969000000000001,"15min":29.899000000000001},"min":{"1min":5.9989999999999997,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":36.935000000000002,"5min":387.15600000000001,"15min":387.15600000000001},"last":20.484999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.308999999999999,"5min":15.284000000000001,"15min":18.398},"min":{"1min":6.6799999999999997,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":16.988,"5min":156.90299999999999,"15min":156.90299999999999},"last":22.667999999999999},{"interface":"front","average":{"1min":15.432,"5min":17.199000000000002,"15min":19.454999999999998},"min":{"1min":11.369999999999999,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":22.655000000000001,"5min":143.578,"15min":143.578},"last":23.745000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":16.132000000000001,"5min":18.914999999999999,"15min":20.946999999999999},"min":{"1min":7.7999999999999998,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":27.901,"5min":141.18799999999999,"15min":141.18799999999999},"last":21.300000000000001},{"interface":"front","average":{"1min":18.728000000000002,"5min":21.913,"15min":24.251000000000001},"min":{"1min":9.1099999999999994,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":35.975999999999999,"5min":161.32900000000001,"15min":161.32900000000001},"last":29.498999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":12.367000000000001,"5min":13.390000000000001,"15min":14.493},"min":{"1min":7.835,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":22.731000000000002,"5min":74.079999999999998,"15min":74.079999999999998},"last":8.5069999999999997},{"interface":"front","average":{"1min":15.497,"5min":15.002000000000001,"15min":15.468},"min":{"1min":7.6379999999999999,"5min":7.6379999999999999,"15min":7.6379999999999999},"max":{"1min":24.989999999999998,"5min":58.613999999999997,"15min":58.613999999999997},"last":25.407}]},{"osd":5,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":13.66,"5min":15.308,"15min":17.079999999999998},"min":{"1min":8.4019999999999992,"5min":7.383,"15min":7.383},"max":{"1min":24.629999999999999,"5min":71.757000000000005,"15min":71.757000000000005},"last":10.41},{"interface":"front","average":{"1min":18.666,"5min":19.806999999999999,"15min":21.109999999999999},"min":{"1min":9.532,"5min":9.532,"15min":9.532},"max":{"1min":29.719999999999999,"5min":80.119,"15min":80.119},"last":24.709}]},{"osd":7,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":17.417000000000002,"15min":18.966000000000001},"min":{"1min":9.9329999999999998,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":23.734999999999999,"5min":81.247,"15min":81.247},"last":11.686999999999999},{"interface":"front","average":{"1min":16.643999999999998,"5min":17.164999999999999,"15min":18.446999999999999},"min":{"1min":8.6310000000000002,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":31.199000000000002,"5min":70.989000000000004,"15min":70.989000000000004},"last":27.321999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476800,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.148999999999999,"5min":10.869999999999999,"15min":10.996},"min":{"1min":4.9450000000000003,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":48.792000000000002,"5min":48.792000000000002,"15min":48.792000000000002},"last":12.890000000000001},{"interface":"front","average":{"1min":11.84,"5min":10.802,"15min":10.439},"min":{"1min":7.0129999999999999,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":28.616,"5min":28.616,"15min":28.616},"last":6.915}]},{"osd":1,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":13.914999999999999,"5min":19.594000000000001,"15min":24.416},"min":{"1min":7.46,"5min":6.4020000000000001,"15min":6.4020000000000001},"max":{"1min":33.198,"5min":165.76400000000001,"15min":165.76400000000001},"last":15.276},{"interface":"front","average":{"1min":13.18,"5min":19.954999999999998,"15min":25.370999999999999},"min":{"1min":7.8289999999999997,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":21.282,"5min":190.34200000000001,"15min":190.34200000000001},"last":12.006}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.190999999999999,"5min":16.754999999999999,"15min":17.870000000000001},"min":{"1min":8.548,"5min":7.96,"15min":7.96},"max":{"1min":34.350000000000001,"5min":41.229999999999997,"15min":41.229999999999997},"last":13.701000000000001},{"interface":"front","average":{"1min":15.375999999999999,"5min":15.984,"15min":16.794},"min":{"1min":8.109,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":47.140999999999998,"5min":47.140999999999998,"15min":47.140999999999998},"last":17.533000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.747,"5min":18.550999999999998,"15min":19.93},"min":{"1min":11.375,"5min":9.8979999999999997,"15min":9.8979999999999997},"max":{"1min":44.091000000000001,"5min":44.091000000000001,"15min":44.091000000000001},"last":16.076000000000001},{"interface":"front","average":{"1min":12.366,"5min":11.036,"15min":11.420999999999999},"min":{"1min":5.9400000000000004,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":45.231000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.2550000000000008}]},{"osd":5,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.690999999999999,"5min":16.504999999999999,"15min":18.606999999999999},"min":{"1min":11.563000000000001,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":49.784999999999997,"5min":56.796999999999997,"15min":56.796999999999997},"last":19.428000000000001},{"interface":"front","average":{"1min":13.282999999999999,"5min":17.286999999999999,"15min":19.628},"min":{"1min":8.0399999999999991,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":28.132999999999999,"5min":58.576000000000001,"15min":58.576000000000001},"last":8.8559999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":15.202,"5min":22.263999999999999,"15min":26.846},"min":{"1min":8.4260000000000002,"5min":8.4260000000000002,"15min":8.4260000000000002},"max":{"1min":29.905000000000001,"5min":169.05199999999999,"15min":169.05199999999999},"last":9.9979999999999993},{"interface":"front","average":{"1min":16.574000000000002,"5min":14.781000000000001,"15min":15.256},"min":{"1min":9.5060000000000002,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":34.969000000000001,"5min":40.128999999999998,"15min":40.128999999999998},"last":10.561}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":19.837,"5min":18.343,"15min":18.105},"min":{"1min":12.76,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":50.722000000000001,"5min":50.722000000000001,"15min":50.722000000000001},"last":21.396999999999998},{"interface":"front","average":{"1min":18.157,"5min":23.681000000000001,"15min":27.637},"min":{"1min":11.978999999999999,"5min":10.025,"15min":10.025},"max":{"1min":29.071999999999999,"5min":170.232,"15min":170.232},"last":19.818000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":9.9510000000000005,"5min":17.408999999999999,"15min":24.341999999999999},"min":{"1min":5.4710000000000001,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":27.117000000000001,"5min":258.66199999999998,"15min":258.66199999999998},"last":15.102},{"interface":"front","average":{"1min":10.667,"5min":15.890000000000001,"15min":21.158000000000001},"min":{"1min":5.3499999999999996,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":28.109999999999999,"5min":203.38499999999999,"15min":203.38499999999999},"last":11.819000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.951000000000001,"5min":14.247999999999999,"15min":15.622999999999999},"min":{"1min":7.468,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":27.626999999999999,"5min":45.936999999999998,"15min":45.936999999999998},"last":13.709},{"interface":"front","average":{"1min":12.977,"5min":13.169,"15min":14.282},"min":{"1min":6.6070000000000002,"5min":6.6070000000000002,"15min":6.6070000000000002},"max":{"1min":21.780999999999999,"5min":37.390999999999998,"15min":37.390999999999998},"last":19.134}]},{"osd":3,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.968999999999999,"5min":14.577,"15min":17.742999999999999},"min":{"1min":6.8899999999999997,"5min":6.5300000000000002,"15min":6.5300000000000002},"max":{"1min":23.518999999999998,"5min":183.38900000000001,"15min":183.38900000000001},"last":6.8019999999999996},{"interface":"front","average":{"1min":16.148,"5min":18.513000000000002,"15min":21.548999999999999},"min":{"1min":7.6399999999999997,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":24.055,"5min":189.374,"15min":189.374},"last":15.715}]},{"osd":4,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":12.247999999999999,"5min":17.015000000000001,"15min":20.311},"min":{"1min":8.3019999999999996,"5min":7.7229999999999999,"15min":7.7229999999999999},"max":{"1min":16.420999999999999,"5min":186.01400000000001,"15min":186.01400000000001},"last":9.2040000000000006},{"interface":"front","average":{"1min":16.623000000000001,"5min":16.381,"15min":18.870000000000001},"min":{"1min":8.9849999999999994,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":36.784999999999997,"5min":167.67599999999999,"15min":167.67599999999999},"last":18.524000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":18.126999999999999,"5min":21.486000000000001,"15min":25.535},"min":{"1min":10.58,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":24.699000000000002,"5min":215.80500000000001,"15min":215.80500000000001},"last":16.844999999999999},{"interface":"front","average":{"1min":17.141999999999999,"5min":20.536000000000001,"15min":25.065999999999999},"min":{"1min":8.8520000000000003,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":33.207999999999998,"5min":187.863,"15min":187.863},"last":13.087}]},{"osd":6,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.503,"5min":20.372,"15min":24.965},"min":{"1min":11.701000000000001,"5min":10.32,"15min":10.32},"max":{"1min":28.986999999999998,"5min":228.81200000000001,"15min":228.81200000000001},"last":7.7969999999999997},{"interface":"front","average":{"1min":20.504999999999999,"5min":23.777000000000001,"15min":28.158000000000001},"min":{"1min":12.313000000000001,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":33.987000000000002,"5min":275.92200000000003,"15min":275.92200000000003},"last":20.614999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":19.542999999999999,"5min":21.734999999999999,"15min":25.396000000000001},"min":{"1min":9.4580000000000002,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":35.898000000000003,"5min":170.43600000000001,"15min":170.43600000000001},"last":17.850999999999999},{"interface":"front","average":{"1min":18.407,"5min":19.861999999999998,"15min":22.312000000000001},"min":{"1min":8.5280000000000005,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.297000000000001,"5min":157.00200000000001,"15min":157.00200000000001},"last":10.759}]}]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":316246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":9.4700000000000006,"5min":10.786,"15min":10.725},"min":{"1min":4.9829999999999997,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":25.042999999999999,"5min":56.704000000000001,"15min":56.704000000000001},"last":10.776},{"interface":"front","average":{"1min":10.782999999999999,"5min":11.834,"15min":11.882999999999999},"min":{"1min":5.8559999999999999,"5min":5.0010000000000003,"15min":5.0010000000000003},"max":{"1min":26.018999999999998,"5min":57.805,"15min":57.805},"last":10.196999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":10.135,"5min":13.997,"15min":18.033000000000001},"min":{"1min":6.5469999999999997,"5min":6.3289999999999997,"15min":6.3289999999999997},"max":{"1min":17.658000000000001,"5min":127.95399999999999,"15min":127.95399999999999},"last":8.7260000000000009},{"interface":"front","average":{"1min":13.305,"5min":13.374000000000001,"15min":14.372},"min":{"1min":7.4390000000000001,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":35.310000000000002,"5min":35.834000000000003,"15min":35.834000000000003},"last":12.269}]},{"osd":3,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.784000000000001,"5min":16.899000000000001,"15min":20.733000000000001},"min":{"1min":8.6069999999999993,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":18.431000000000001,"5min":99.790999999999997,"15min":99.790999999999997},"last":20.157},{"interface":"front","average":{"1min":15.534000000000001,"5min":15.677,"15min":16.369},"min":{"1min":10.504,"5min":9.4689999999999994,"15min":9.4689999999999994},"max":{"1min":43.860999999999997,"5min":43.860999999999997,"15min":43.860999999999997},"last":21.922999999999998}]},{"osd":4,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":14.250999999999999,"5min":14.385999999999999,"15min":15.895},"min":{"1min":8.2260000000000009,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":51.802,"5min":51.802,"15min":51.802},"last":7.1559999999999997},{"interface":"front","average":{"1min":18.021000000000001,"5min":19.768000000000001,"15min":23.504999999999999},"min":{"1min":10.582000000000001,"5min":7.6689999999999996,"15min":7.6689999999999996},"max":{"1min":40.000999999999998,"5min":103.705,"15min":103.705},"last":14.912000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":12.994,"5min":15.896000000000001,"15min":17.991},"min":{"1min":9.1519999999999992,"5min":7.8330000000000002,"15min":7.8330000000000002},"max":{"1min":17.576000000000001,"5min":43.847000000000001,"15min":43.847000000000001},"last":12.898999999999999},{"interface":"front","average":{"1min":19.388000000000002,"5min":21.844999999999999,"15min":26.18},"min":{"1min":9.3119999999999994,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":40.598999999999997,"5min":98.707999999999998,"15min":98.707999999999998},"last":18.882000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.209,"5min":19.010999999999999,"15min":20.006},"min":{"1min":11.734,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":51.018999999999998,"5min":55.631,"15min":55.631},"last":16.119},{"interface":"front","average":{"1min":16.334,"5min":19.213000000000001,"15min":21.271000000000001},"min":{"1min":9.907,"5min":9.7159999999999993,"15min":9.7159999999999993},"max":{"1min":48.860999999999997,"5min":54.656999999999996,"15min":54.656999999999996},"last":17.809000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:19 2023","interfaces":[{"interface":"back","average":{"1min":19.692,"5min":19.841000000000001,"15min":21.363},"min":{"1min":12.294,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":53.314,"5min":53.314,"15min":53.314},"last":22.963999999999999},{"interface":"front","average":{"1min":18.108000000000001,"5min":20.082000000000001,"15min":22.216000000000001},"min":{"1min":10.589,"5min":10.589,"15min":10.589},"max":{"1min":37.427999999999997,"5min":45.130000000000003,"15min":45.130000000000003},"last":15.539999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":332857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.51,"5min":12.632999999999999,"15min":15.106999999999999},"min":{"1min":4.944,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":40.948,"5min":131.477,"15min":131.477},"last":16.724},{"interface":"front","average":{"1min":11.792999999999999,"5min":13.177,"15min":15.381},"min":{"1min":6.4649999999999999,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":40.159999999999997,"5min":129.386,"15min":129.386},"last":7.0339999999999998}]},{"osd":1,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":10.955,"5min":10.936999999999999,"15min":10.975},"min":{"1min":6.8940000000000001,"5min":6.3390000000000004,"15min":6.3390000000000004},"max":{"1min":18.311,"5min":25.774000000000001,"15min":25.774000000000001},"last":11.619},{"interface":"front","average":{"1min":15.336,"5min":17.414999999999999,"15min":18.797000000000001},"min":{"1min":12.584,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":19.048999999999999,"5min":91.295000000000002,"15min":91.295000000000002},"last":19.695}]},{"osd":2,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":11.068,"5min":14.746,"15min":17.061},"min":{"1min":6.5110000000000001,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":21.138999999999999,"5min":134.21199999999999,"15min":134.21199999999999},"last":17.588000000000001},{"interface":"front","average":{"1min":12.34,"5min":14.003,"15min":14.423999999999999},"min":{"1min":7.3540000000000001,"5min":5.992,"15min":5.992},"max":{"1min":22.401,"5min":50.450000000000003,"15min":50.450000000000003},"last":14.801}]},{"osd":3,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":14.092000000000001,"5min":16.588999999999999,"15min":19.309000000000001},"min":{"1min":7.2750000000000004,"5min":6.758,"15min":6.758},"max":{"1min":22.478999999999999,"5min":132.56100000000001,"15min":132.56100000000001},"last":16.347999999999999},{"interface":"front","average":{"1min":16.396999999999998,"5min":17.763999999999999,"15min":20.045999999999999},"min":{"1min":11.826000000000001,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":22.414999999999999,"5min":124.684,"15min":124.684},"last":21.623000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":14.701000000000001,"5min":13.255000000000001,"15min":13.677},"min":{"1min":10.32,"5min":6.6050000000000004,"15min":6.6050000000000004},"max":{"1min":21.858000000000001,"5min":24.963000000000001,"15min":24.963000000000001},"last":20.289999999999999},{"interface":"front","average":{"1min":17.369,"5min":16.047000000000001,"15min":15.75},"min":{"1min":10.175000000000001,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":28.701000000000001,"5min":28.701000000000001,"15min":28.701000000000001},"last":22.411000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:18 2023","interfaces":[{"interface":"back","average":{"1min":11.826000000000001,"5min":16.309999999999999,"15min":19.253},"min":{"1min":6.9550000000000001,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":20.228000000000002,"5min":125.792,"15min":125.792},"last":14.304},{"interface":"front","average":{"1min":16.962,"5min":16.856000000000002,"15min":16.956},"min":{"1min":12.593999999999999,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":28.417999999999999,"5min":28.591999999999999,"15min":28.591999999999999},"last":29.338000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:24 2023","interfaces":[{"interface":"back","average":{"1min":18.372,"5min":22.959,"15min":29.097999999999999},"min":{"1min":13.268000000000001,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":30.768999999999998,"5min":327.29599999999999,"15min":327.29599999999999},"last":33.497},{"interface":"front","average":{"1min":13.772,"5min":22.376000000000001,"15min":30.071000000000002},"min":{"1min":8.5779999999999994,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":25.986999999999998,"5min":323.82400000000001,"15min":323.82400000000001},"last":20.809000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":730697,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":8.4239999999999995,"5min":11.695,"15min":13.956},"min":{"1min":4.2569999999999997,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":16.850000000000001,"5min":68.513999999999996,"15min":68.513999999999996},"last":45.569000000000003},{"interface":"front","average":{"1min":9.6069999999999993,"5min":12.859999999999999,"15min":15.647},"min":{"1min":6.8209999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":14.297000000000001,"5min":88.816000000000003,"15min":88.816000000000003},"last":42.341999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":12.551,"5min":15.057,"15min":16.774999999999999},"min":{"1min":6.2869999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":21.32,"5min":87.515000000000001,"15min":87.515000000000001},"last":6.9870000000000001},{"interface":"front","average":{"1min":13.327,"5min":15.451000000000001,"15min":17.158999999999999},"min":{"1min":8.8840000000000003,"5min":6.21,"15min":6.21},"max":{"1min":22.943000000000001,"5min":85.832999999999998,"15min":85.832999999999998},"last":14.896000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":14.614000000000001,"5min":16.452000000000002,"15min":17.725999999999999},"min":{"1min":6.5259999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":27.128,"5min":113.943,"15min":113.943},"last":23.059999999999999},{"interface":"front","average":{"1min":10.391,"5min":12.885999999999999,"15min":14.608000000000001},"min":{"1min":6.2889999999999997,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.457999999999998,"5min":80.004000000000005,"15min":80.004000000000005},"last":9.6760000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":15.539,"5min":18.071999999999999,"15min":20.584},"min":{"1min":10.071999999999999,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":21.744,"5min":87.866,"15min":87.866},"last":10.471},{"interface":"front","average":{"1min":15.82,"5min":19.992999999999999,"15min":22.632000000000001},"min":{"1min":8.4550000000000001,"5min":8.4550000000000001,"15min":8.4550000000000001},"max":{"1min":28.135000000000002,"5min":91.183000000000007,"15min":91.183000000000007},"last":25.966000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:28:16 2023","interfaces":[{"interface":"back","average":{"1min":11.984,"5min":12.398999999999999,"15min":12.625},"min":{"1min":6.7990000000000004,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":24.067,"5min":25.195,"15min":25.195},"last":15.763},{"interface":"front","average":{"1min":17.079999999999998,"5min":16.25,"15min":16.873999999999999},"min":{"1min":10.51,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":31.893000000000001,"5min":59.609000000000002,"15min":59.609000000000002},"last":12.773999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.930999999999999,"5min":15.021000000000001,"15min":15.755000000000001},"min":{"1min":7.9820000000000002,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":30.594000000000001,"5min":66.765000000000001,"15min":66.765000000000001},"last":13.481999999999999},{"interface":"front","average":{"1min":18.491,"5min":19.263000000000002,"15min":20.593},"min":{"1min":13.439,"5min":9.4399999999999995,"15min":9.4399999999999995},"max":{"1min":33.707000000000001,"5min":71.771000000000001,"15min":71.771000000000001},"last":7.8220000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:28:15 2023","interfaces":[{"interface":"back","average":{"1min":17.425000000000001,"5min":18.123999999999999,"15min":18.806000000000001},"min":{"1min":10.709,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":29.001999999999999,"5min":64.677000000000007,"15min":64.677000000000007},"last":24.724},{"interface":"front","average":{"1min":15.053000000000001,"5min":16.744,"15min":17.835000000000001},"min":{"1min":9.1760000000000002,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":20.460999999999999,"5min":42.531999999999996,"15min":42.531999999999996},"last":11.250999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":332857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":9.9199999999999999,"5min":9.6159999999999997,"15min":9.4120000000000008},"min":{"1min":4.9379999999999997,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":16.841000000000001,"5min":26.984999999999999,"15min":26.984999999999999},"last":6.2930000000000001},{"interface":"front","average":{"1min":10.007999999999999,"5min":10.661,"15min":10.592000000000001},"min":{"1min":5.2119999999999997,"5min":5.2119999999999997,"15min":5.2119999999999997},"max":{"1min":15.428000000000001,"5min":25.934999999999999,"15min":25.934999999999999},"last":14.298999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:28:21 2023","interfaces":[{"interface":"back","average":{"1min":13.516999999999999,"5min":12.371,"15min":11.771000000000001},"min":{"1min":7.0449999999999999,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":25.122,"5min":25.122,"15min":25.122},"last":7.5129999999999999},{"interface":"front","average":{"1min":12.935,"5min":13.680999999999999,"15min":13.994999999999999},"min":{"1min":8.3219999999999992,"5min":6.609,"15min":6.609},"max":{"1min":20.672999999999998,"5min":30.751999999999999,"15min":30.751999999999999},"last":16.236000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":12.712999999999999,"5min":13.198,"15min":12.378},"min":{"1min":7.5339999999999998,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":19.831,"5min":25.844000000000001,"15min":25.844000000000001},"last":17.111000000000001},{"interface":"front","average":{"1min":14.474,"5min":15.125999999999999,"15min":15.558999999999999},"min":{"1min":9.2080000000000002,"5min":9.1539999999999999,"15min":9.1539999999999999},"max":{"1min":23.298999999999999,"5min":25.428000000000001,"15min":25.428000000000001},"last":10.054}]},{"osd":3,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":8.4220000000000006,"5min":10.686,"15min":11.071999999999999},"min":{"1min":6.1260000000000003,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":12.452999999999999,"5min":66.722999999999999,"15min":66.722999999999999},"last":12.023},{"interface":"front","average":{"1min":15.635,"5min":16.170999999999999,"15min":16.100999999999999},"min":{"1min":8.1050000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"max":{"1min":27.145,"5min":69.346000000000004,"15min":69.346000000000004},"last":13.960000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.201000000000001,"5min":18.181000000000001,"15min":19.129000000000001},"min":{"1min":9.4939999999999998,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":25.771999999999998,"5min":64.474999999999994,"15min":64.474999999999994},"last":17.998999999999999},{"interface":"front","average":{"1min":12.016,"5min":14.678000000000001,"15min":15.699999999999999},"min":{"1min":6.6440000000000001,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":32.332999999999998,"5min":75.980000000000004,"15min":75.980000000000004},"last":8.8940000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":18.664999999999999,"5min":18.544,"15min":18.699999999999999},"min":{"1min":12.724,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":38.712000000000003,"5min":38.712000000000003,"15min":38.712000000000003},"last":18.602},{"interface":"front","average":{"1min":14.061999999999999,"5min":13.598000000000001,"15min":13.882999999999999},"min":{"1min":8.8849999999999998,"5min":7.343,"15min":7.343},"max":{"1min":33.802999999999997,"5min":38.668999999999997,"15min":38.668999999999997},"last":11.407}]},{"osd":7,"last update":"Mon Oct 30 06:28:17 2023","interfaces":[{"interface":"back","average":{"1min":16.358000000000001,"5min":17.379999999999999,"15min":18.032},"min":{"1min":11.266,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":22.605,"5min":74.977999999999994,"15min":74.977999999999994},"last":14.977},{"interface":"front","average":{"1min":18.617999999999999,"5min":20.350000000000001,"15min":21.686},"min":{"1min":14.305999999999999,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":29.989000000000001,"5min":83.138000000000005,"15min":83.138000000000005},"last":19.539000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:29:12.281 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [3, 0, 5, 2] 2023-10-30T06:29:12.281 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '3', '0', '5', '2'] 2023-10-30T06:29:12.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 3 0 5 2 2023-10-30T06:29:12.306 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:12.306 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:12.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:12.361 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:12.360+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:12.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:29:12.474 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:12.473+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:12.489 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:12.489 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:12.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:12.583 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:12.582+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:12.666 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:12.667 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:12.671 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:12.669+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:12.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:12.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2023-10-30T06:29:12.765 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:12.763+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:12.863 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:12.864 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:12.865 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:12.865+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:12.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:12.966 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:12.965+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:13.062 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.062 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.067 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:13.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:13.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:13.167 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:13.167+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:13.244 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.244 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:13.297 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:13.294+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:13.369 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:13.368+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.423 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:13.475 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:13.473+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:13.571 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:13.570+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:13.604 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.605 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:13.698 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:13.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:13.773 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:13.771+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:13.782 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.782 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:13.863 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:29:13.873 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:13.872+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:13.960 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.0 pg_upmap_items mapping to [3->0,5->2] 2023-10-30T06:29:13.974 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:13.974+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:13.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2023-10-30T06:29:13.989 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:13.989 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:13.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:14.075 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:14.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:14.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:14.134 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:14.135 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:14.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:14.188 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:14.187+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:14.270 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:14.270 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:14.277 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:14.276+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:14.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:14.341 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:14.341 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.379 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:14.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:14.430 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:14.431 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.449 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:14.449 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:14.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:14.499 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:14.498+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:14.531 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:14.531 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.580 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:14.578+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:14.623 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:14.624 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:14.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:14.633 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:14.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.680 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:14.679+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:14.742 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:14.742 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.781 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:14.780+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:14.843 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:14.844 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.844 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:14.844 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:14.845 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:14.845 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:14.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:14.882 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:14.880+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:14.944 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:14.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2023-10-30T06:29:14.982 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:14.981+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:15.023 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:15.023 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:15.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:15.023 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:15.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.024 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:15.024 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:15.024 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 302.865512868, 2023-10-30T06:29:15.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:15.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:15.025 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:15.025 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:15.026 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:15.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:15.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.026 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:15.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:15.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:15.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:15.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:15.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:15.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:15.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:15.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:15.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:15.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:15.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:15.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:15.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:15.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:15.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:15.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:15.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.040 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.041 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:15.041 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:15.041 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 299.952339005, 2023-10-30T06:29:15.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:15.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.042 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:15.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:15.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:15.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:15.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:15.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:15.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:15.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:15.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:15.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:15.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:15.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:15.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:15.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:15.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:15.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:15.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:15.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:15.056 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.056 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.057 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:15.057 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:15.058 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 299.29216103599998, 2023-10-30T06:29:15.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:15.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.058 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:15.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:15.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:15.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:15.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:15.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:15.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:15.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:15.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:15.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:15.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:15.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:15.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:15.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 299.21673782900001, 2023-10-30T06:29:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:15.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.071 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:15.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:15.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:15.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:15.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:15.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:15.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.077 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:15.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:15.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.078 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:15.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:15.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:15.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:15.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:15.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:15.082 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.082 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.082 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.083 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:15.083 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.94561469799999, 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:15.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:15.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:15.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:15.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:15.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:15.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:15.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:15.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:15.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:15.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:15.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:15.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:15.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:15.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:15.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:15.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:15.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:15.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.095 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.91533498799998, 2023-10-30T06:29:15.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:15.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.097 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:15.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:15.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:15.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:15.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:15.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:15.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:15.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:15.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:15.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:15.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:15.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:15.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:15.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:15.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.107 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.108 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:15.108 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:15.108 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.541423904, 2023-10-30T06:29:15.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:15.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:15.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:15.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:15.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:15.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:15.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:15.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:15.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:15.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:15.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:15.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:15.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:15.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:15.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:15.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:15.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:15.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.119 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.119 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.120 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:15.120 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:15.120 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.53692520300001, 2023-10-30T06:29:15.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:15.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:15.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:15.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:15.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:15.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:15.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:15.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:15.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:15.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:15.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:15.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:15.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:15.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:15.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:15.130 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.131 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.131 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:15.132 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:15.132 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.24532126399998, 2023-10-30T06:29:15.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:15.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.133 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:15.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:15.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:15.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:15.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:15.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:15.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:15.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:15.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:15.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:15.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:15.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:15.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:15.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:15.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:15.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:15.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:15.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:15.145 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.145 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.146 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.146 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:15.146 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.23938394700002, 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:15.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:15.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:15.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:15.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:15.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:15.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:15.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:15.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:15.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:15.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:15.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:15.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:15.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:15.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:15.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:15.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.158 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.158 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.158 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.23808493199999, 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.159 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:15.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:15.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:15.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:15.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:15.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:15.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:15.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:15.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:15.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:15.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:15.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:15.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:15.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:15.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:15.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:15.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:15.169 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.170 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.170 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.171 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:15.171 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:15.171 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.13509612399997, 2023-10-30T06:29:15.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:15.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.172 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:15.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:15.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:15.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:15.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:15.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:15.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:15.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:15.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:15.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:15.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:15.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:15.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:15.182 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.182 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.182 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.183 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:15.183 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:15.183 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.863192857, 2023-10-30T06:29:15.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:15.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.184 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:15.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:15.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:15.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:15.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:15.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:15.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:15.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:15.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:15.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:15.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:15.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:15.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:15.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:15.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:15.194 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.195 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.195 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.85569466499999, 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.196 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:15.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:15.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:15.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:15.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:15.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:15.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:15.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:15.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:15.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:15.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:15.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:15.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:15.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:15.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.207 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.207 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.85503346000002, 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.208 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:15.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:15.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:15.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:15.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:15.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:15.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:15.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:15.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:15.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:15.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:15.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:15.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:15.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:15.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:15.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.218 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.219 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:15.219 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:15.220 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.84244762200001, 2023-10-30T06:29:15.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:15.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.220 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:15.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:15.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:15.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:15.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:15.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:15.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:15.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:15.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:15.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:15.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:15.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:15.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:15.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.230 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.83635540900002, 2023-10-30T06:29:15.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:15.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.232 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:15.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:15.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:15.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:15.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:15.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:15.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:15.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:15.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:15.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:15.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:15.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:15.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:15.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:15.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.242 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.243 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:15.243 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:15.243 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 297.835360068, 2023-10-30T06:29:15.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:15.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:15.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:15.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:15.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:15.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:15.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:15.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:15.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:15.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:15.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:15.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:15.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:15.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:15.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:15.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:15.254 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.254 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.254 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.255 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:15.255 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:15.255 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 290.512687497, 2023-10-30T06:29:15.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:15.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:15.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:15.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:15.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:15.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:15.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:15.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:15.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:15.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:15.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:15.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:15.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:15.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:15.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:15.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:15.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:15.265 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.266 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 290.47924601199998, 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:15.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:15.268 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:15.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:15.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:15.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:15.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:15.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:15.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:15.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:15.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:15.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:15.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:15.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:15.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:15.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:15.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:15.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:15.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:15.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:15.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:15.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:15.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:15.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:15.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:15.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:15.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:15.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:15.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:15.277 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.278 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.278 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.278 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:15.278 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:15.278 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:15.279 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:15.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:15.281 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:15.282 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:15.278+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:15.381 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:15.379+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:15.449 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:15.450 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:15.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:15.482 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:15.480+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:15.583 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:15.582+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:15.626 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:15.626 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:15.627 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:15.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:15.684 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:15.682+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:15.782 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:15.783 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:15.784 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:15.783+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:15.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:15.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:29:15.885 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:15.883+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:15.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2023-10-30T06:29:15.921 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:15.922 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:15.922 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:15.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:16.016 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:16.014+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:16.086 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:16.085+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:16.096 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:16.098 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:16.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:16.190 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:16.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:16.276 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:16.277 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:16.278 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 219.30284226200001, 2023-10-30T06:29:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:16.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:16.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:16.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:16.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:16.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:16.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:16.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.290 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 39.282349938000003, 2023-10-30T06:29:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:16.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:16.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:16.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:16.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:16.301 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:16.302 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:16.302 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:16.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:16.304 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:16.302+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:16.405 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:16.403+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:16.448 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:16.448 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:16.448 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:16.448 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:16.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:16.506 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:16.504+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:16.606 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:16.604+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:16.628 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:16.628 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:16.628 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:16.628 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:16.629 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:16.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:16.707 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:16.706+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:16.808 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:16.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:16.817 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:16.817 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:16.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:16.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:16.818 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:16.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:16.910 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:16.908+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:17.001 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:17.001 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:17.001 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:17.001 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:17.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:17.011 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:17.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:29:17.110 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:17.109+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:17.150 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:17.151 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:17.151 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:17.151 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:17.151 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:17.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:17.245 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:17.243+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:17.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:17.312 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:17.311+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:17.393 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:17.394 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:17.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:17.394 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:17.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 304.047344125, 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:17.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.397 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:17.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:17.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:17.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:17.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:17.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:17.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:17.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:17.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:17.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:17.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:17.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:17.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:17.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:17.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:17.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:17.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:17.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:17.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:17.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:17.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:17.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:17.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:17.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:17.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.413 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:17.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.72618893399999, 2023-10-30T06:29:17.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:17.414 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:17.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:17.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:17.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:17.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:17.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:17.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:17.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:17.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:17.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:17.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:17.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:17.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:17.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:17.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:17.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:17.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:17.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:17.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:17.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:17.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.429 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.429 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.721944377, 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.431 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.431 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:17.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.432 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:17.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:17.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:17.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:17.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:17.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:17.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:17.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:17.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:17.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:17.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:17.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:17.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:17.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:17.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:17.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:17.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:17.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:17.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:17.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:17.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:17.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:17.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:17.444 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.445 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.445 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.445 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.446 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:17.446 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.63648595199999, 2023-10-30T06:29:17.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:17.446 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.446 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:17.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:17.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:17.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:17.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:17.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:17.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:17.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:17.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:17.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:17.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:17.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:17.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:17.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:17.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:17.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:17.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:17.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:17.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:17.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:17.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:17.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:17.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:17.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:17.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.36724273599998, 2023-10-30T06:29:17.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:17.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:17.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:17.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:17.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:17.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:17.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:17.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:17.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:17.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:17.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:17.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:17.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:17.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:17.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:17.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:17.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:17.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:17.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:17.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:17.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:17.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:17.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:17.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.366411325, 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:17.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.480 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:17.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:17.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:17.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:17.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:17.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:17.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:17.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:17.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:17.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:17.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:17.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:17.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:17.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:17.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:17.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:17.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.493 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:17.494 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.36574419800002, 2023-10-30T06:29:17.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:17.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:17.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.496 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:17.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:17.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:17.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:17.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:17.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:17.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:17.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:17.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:17.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:17.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:17.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:17.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:17.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:17.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:17.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:17.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:17.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:17.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:17.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:17.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:17.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:17.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.510 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.510 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:17.510 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.34347012699999, 2023-10-30T06:29:17.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:17.511 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.511 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.511 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.512 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:17.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.512 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:17.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:17.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:17.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:17.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:17.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:17.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:17.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:17.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:17.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:17.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:17.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:17.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:17.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:17.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:17.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:17.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:17.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:17.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:17.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:17.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:17.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:17.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.525 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.525 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.526 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:17.526 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.34282227099999, 2023-10-30T06:29:17.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:17.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.527 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.528 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:17.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.528 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:17.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:17.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:17.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:17.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:17.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:17.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:17.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:17.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:17.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:17.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:17.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:17.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:17.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:17.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:17.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:17.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 301.34213619799999, 2023-10-30T06:29:17.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:17.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:17.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.544 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:17.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:17.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:17.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:17.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:17.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:17.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:17.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:17.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:17.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:17.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:17.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:17.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:17.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:17.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:17.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:17.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:17.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:17.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:17.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:17.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.558 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:17.558 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:17.558 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.27125423000001, 2023-10-30T06:29:17.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:17.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.559 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.559 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.560 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:17.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.560 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:17.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:17.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:17.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:17.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:17.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:17.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:17.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:17.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:17.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:17.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:17.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:17.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:17.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:17.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:17.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:17.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:17.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:17.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.574 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:17.574 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:17.574 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.27059026400002, 2023-10-30T06:29:17.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:17.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.575 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.575 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.575 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.575 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.575 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:17.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.576 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:17.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:17.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:17.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:17.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:17.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:17.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:17.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:17.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:17.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:17.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:17.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:17.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:17.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:17.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:17.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:17.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.589 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.589 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:17.590 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:17.590 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 300.24213645499998, 2023-10-30T06:29:17.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:17.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.591 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.591 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:17.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.592 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:17.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:17.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:17.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:17.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:17.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:17.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:17.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:17.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:17.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:17.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:17.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:17.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:17.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:17.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:17.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:17.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:17.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:17.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:17.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:17.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:17.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:17.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:17.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.606 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.606 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:17.606 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.90036996200001, 2023-10-30T06:29:17.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:17.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:17.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.608 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:17.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:17.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:17.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:17.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:17.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:17.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:17.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:17.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:17.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:17.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:17.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:17.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:17.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:17.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:17.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:17.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:17.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:17.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:17.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:17.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:17.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:17.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.621 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.622 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:17.622 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.89962129200001, 2023-10-30T06:29:17.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:17.622 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.622 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.623 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:17.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:17.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:17.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:17.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:17.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:17.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:17.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:17.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:17.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:17.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:17.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:17.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:17.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:17.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:17.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:17.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:17.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:17.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:17.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:17.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:17.636 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.637 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.637 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.638 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:17.638 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.898952909, 2023-10-30T06:29:17.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:17.638 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.638 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:17.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.640 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:17.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:17.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:17.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:17.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:17.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:17.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:17.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:17.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:17.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:17.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:17.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:17.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:17.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:17.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:17.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:17.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:17.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:17.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:17.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:17.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:17.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:17.652 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.653 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.654 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:17.654 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.889102235, 2023-10-30T06:29:17.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:17.654 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.654 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.655 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.655 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:17.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:17.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:17.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:17.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:17.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:17.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:17.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:17.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:17.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:17.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:17.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:17.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:17.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:17.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:17.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:17.668 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.669 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.88842842000003, 2023-10-30T06:29:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.670 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:17.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:17.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:17.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:17.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:17.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:17.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:17.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:17.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:17.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.684 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:17.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.88778003700003, 2023-10-30T06:29:17.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:17.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:17.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:17.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:17.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:17.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:17.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:17.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:17.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:17.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.883663571, 2023-10-30T06:29:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:17.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:17.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:17.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:17.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:17.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:17.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:17.717 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:17.717 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:17.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:17.719 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:17.720 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:17.720 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:17.721 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:17.721 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:17.721 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:17.727 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:17.726+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:17.789 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:17.790 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:17.820 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:17.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:17.850 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:17.850 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:17.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:17.875 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:17.921 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:17.920+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:17.960 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:17.961 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:18.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2023-10-30T06:29:18.011 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.011 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:18.023 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:18.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:18.049 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:18.050 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:18.123 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:18.122+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:18.149 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:18.149 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:18.187 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.187 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:18.239 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:18.236+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:18.326 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:18.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:18.363 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.417 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:18.459 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:18.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:18.527 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:18.526+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:18.594 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.594 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:18.648 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:18.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:18.729 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:18.728+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:18.745 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.745 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:18.830 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:18.829+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:18.881 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:18.882 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:18.891 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:18.931 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:18.930+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:18.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2023-10-30T06:29:18.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:18.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:19.044 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:19.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:19.058 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.059 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:19.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:29:19.133 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:19.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:19.194 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.197 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.204 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:19.233 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:19.232+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:19.334 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:19.333+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:19.368 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.368 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:19.436 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:19.434+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:19.504 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.505 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:19.559 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:19.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:19.638 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:19.636+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:19.641 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.641 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:19.738 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:19.737+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:19.820 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.821 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:19.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:19.872 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:19.870+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:19.940 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:19.939+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:19.997 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:19.998 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:20.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:20.048 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:20.047+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:20.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2023-10-30T06:29:20.057 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:29:20.142 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:20.140+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:20.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:20.172 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:20.173 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:20.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:20.243 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:20.242+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:20.308 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:20.308 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:20.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:20.358 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:20.357+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:20.417 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.442 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:20.443 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:20.445 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:20.444+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:20.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:20.503 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:20.504 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.546 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:20.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:20.578 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:20.578 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:20.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:20.593 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:20.594 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.648 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:20.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:20.697 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:20.698 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.731 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:20.732 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:20.732 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:20.732 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:20.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:20.749 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:20.748+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:20.797 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:20.798 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.850 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:20.849+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:20.897 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:20.898 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:20.921 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:20.921 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:20.922 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:20.922 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:20.931 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:20.972 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:20.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:20.980 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:20.980 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:21.052 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:21.050+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:21.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2023-10-30T06:29:21.076 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:21.077 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:21.110 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:21.110 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:21.111 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:21.119 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:21.161 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:21.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:21.254 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:21.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:21.305 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:21.306 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:21.306 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:21.306 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:21.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:21.355 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:21.354+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:21.456 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:21.455+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:21.502 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:21.503 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:21.503 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:21.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:21.557 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:21.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:21.658 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:21.656+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:21.741 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:21.741 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:21.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:21.741 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:21.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:21.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:21.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 308.40170348700002, 2023-10-30T06:29:21.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:21.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:21.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:21.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:21.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:21.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:21.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:21.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:21.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:21.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:21.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:21.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:21.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:21.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:21.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:21.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:21.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:21.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:21.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:21.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:21.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:21.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:21.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:21.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:21.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:21.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:21.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.764 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.764 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:21.764 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 306.08054829600002, 2023-10-30T06:29:21.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:21.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:21.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:21.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:21.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:21.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:21.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:21.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:21.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:21.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:21.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:21.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:21.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:21.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:21.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:21.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:21.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:21.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:21.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:21.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.780 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.780 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:21.780 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 306.07630373900002, 2023-10-30T06:29:21.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:21.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.782 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:21.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:21.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:21.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:21.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:21.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:21.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:21.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:21.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:21.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:21.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:21.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:21.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:21.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:21.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:21.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:21.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.99084531400001, 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.797 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.798 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:21.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:21.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:21.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:21.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:21.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:21.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:21.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:21.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:21.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:21.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:21.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:21.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:21.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:21.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:21.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.72160209800001, 2023-10-30T06:29:21.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:21.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:21.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.816 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:21.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:21.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:21.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:21.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:21.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:21.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:21.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:21.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:21.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:21.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:21.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:21.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:21.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:21.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:21.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:21.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:21.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:21.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:21.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:21.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:21.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:21.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:21.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:21.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:21.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.72077068700003, 2023-10-30T06:29:21.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:21.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.834 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.835 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.835 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:21.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.836 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:21.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:21.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:21.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:21.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:21.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:21.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:21.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:21.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:21.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:21.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:21.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:21.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:21.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:21.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.847 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.847 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:21.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:21.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.848 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.848 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:21.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:21.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:21.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:21.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:21.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:21.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:21.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:21.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.852 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.852 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:21.853 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.72010355999998, 2023-10-30T06:29:21.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:21.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.854 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:21.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.855 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:21.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:21.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:21.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:21.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:21.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:21.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:21.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:21.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:21.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:21.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:21.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:21.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:21.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.69782948900001, 2023-10-30T06:29:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:21.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.873 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:21.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:21.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:21.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:21.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:21.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:21.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.886 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.69718163300001, 2023-10-30T06:29:21.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.888 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:21.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:21.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:21.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.904 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 305.69649556000002, 2023-10-30T06:29:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.905 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:21.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:21.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:21.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 304.62561359199998, 2023-10-30T06:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:21.920 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:21.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:21.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:21.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:21.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:21.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:21.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 304.62494962599999, 2023-10-30T06:29:21.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.936 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:21.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:21.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:21.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:21.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:21.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:21.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.949 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:21.950 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 304.596495817, 2023-10-30T06:29:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.951 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:21.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:21.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:21.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:21.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:21.964 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.965 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.25472932399998, 2023-10-30T06:29:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.966 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.967 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:21.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.980 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:21.981 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.25398065399997, 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.982 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:21.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:21.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:21.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:21.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:21.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:21.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:21.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.995 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:21.996 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.25331227100003, 2023-10-30T06:29:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:21.997 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:21.998 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:21.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.011 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:22.012 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.24346159700002, 2023-10-30T06:29:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:22.013 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:22.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:22.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:22.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:22.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:22.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:22.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:22.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:22.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:22.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:22.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:22.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:22.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:22.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:22.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:22.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:22.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:22.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:22.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:22.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:22.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:22.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:22.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:22.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:22.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:22.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:22.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:22.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.24278778199999, 2023-10-30T06:29:22.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:22.029 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:22.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:22.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:22.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:22.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:22.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:22.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:22.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:22.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:22.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:22.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:22.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:22.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:22.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:22.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:22.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:22.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:22.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:22.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:22.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:22.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:22.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:22.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:22.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:22.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:22.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:22.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.042 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.043 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:22.043 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:22.043 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.242139399, 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:22.044 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:22.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:22.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:22.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:22.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:22.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:22.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:22.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:22.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:22.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:22.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:22.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:22.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:22.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:22.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:22.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:22.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:22.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:22.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:22.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:22.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:22.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:22.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:22.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:22.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:22.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:22.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:22.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:22.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:22.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:22.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:22.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:22.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:22.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:22.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.23802293300002, 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:22.059 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:22.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:22.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:22.060 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:22.060 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:22.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.061 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:22.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:22.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:22.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:22.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:22.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:22.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:22.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:22.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:22.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:22.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:22.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:22.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:22.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:22.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:22.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:22.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:22.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:22.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:22.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:22.070 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.070 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.070 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:22.071 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:22.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:22.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:22.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:22.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:22.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:22.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:22.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.073 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.073 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:22.074 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:22.074 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:29:22.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:22.076 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:22.074+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:22.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2023-10-30T06:29:22.177 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:22.175+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:22.264 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:22.264 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:22.264 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:22.265 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:22.278 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:22.276+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:22.379 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:22.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:22.408 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:22.409 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:22.409 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:22.480 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:22.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:22.557 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:22.557 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:22.558 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:22.558 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:22.582 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:22.580+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:22.682 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:22.681+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:22.755 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:22.756 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:22.783 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:22.782+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:22.884 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:22.883+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:22.896 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:22.897 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:22.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:22.986 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:22.984+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:23.076 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:23.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:23.077 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 226.78240608499999, 2023-10-30T06:29:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:23.078 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:23.079 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:23.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:23.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:23.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 46.750231866, 2023-10-30T06:29:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:23.090 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:23.091 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.092 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:23.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:23.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:23.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:23.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:23.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:23.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:23.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:23.104 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:23.105 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:23.105 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:23.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:23.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:23.108 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:23.105+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:23.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2023-10-30T06:29:23.208 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:23.206+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:23.245 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:23.246 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:23.246 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:23.246 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:23.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:29:23.309 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:23.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:23.382 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:23.382 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:23.383 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:23.384 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:23.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:23.432 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:23.431+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:23.437 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:23.437 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:23.510 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:23.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:23.520 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:23.521 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:23.561 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:23.561 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:23.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:23.562 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:23.562 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:23.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:23.613 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:23.611+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:23.644 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:23.645 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:23.724 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:23.722+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:23.737 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:23.738 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:23.738 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:23.738 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:23.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:23.766 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:23.767 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:23.813 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:23.811+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:23.885 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:23.914 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:23.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:23.962 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:23.963 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:23.963 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:23.963 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:23.963 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:23.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:23.979 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:23.979 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:23.980 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-10-30T06:29:23.980 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.225654 2023-10-30T06:29:23.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0.22565418088492806 2023-10-30T06:29:24.024 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:24.023+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:24.026 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:24.026 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:24.106 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.107 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:24.117 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:24.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:24.133 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:24.133 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:24.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2023-10-30T06:29:24.219 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:24.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:24.222 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:24.218+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:24.245 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.245 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.254 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:24.317 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:24.316+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:24.379 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.379 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:24.418 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:24.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:24.513 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.513 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.519 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:24.518+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:24.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:24.620 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:24.618+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:24.648 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.648 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:24.721 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:24.719+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:24.783 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.784 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:24.822 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:24.820+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:24.919 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:24.919 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:24.923 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:24.922+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:24.929 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:25.024 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:25.023+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:25.057 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.058 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:25.125 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:25.124+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:25.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2023-10-30T06:29:25.216 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.216 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:25.227 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:25.224+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:25.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:29:25.313 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 0.225654 (8147882) 2023-10-30T06:29:25.326 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:25.324+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:25.425 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.425 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.426 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:25.425+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:25.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:25.527 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:25.525+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:25.603 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.603 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:25.654 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:25.653+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:25.728 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:25.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:25.791 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.791 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:25.829 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:25.827+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:25.929 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:25.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:25.972 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:25.973 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:25.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:26.030 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:26.029+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:26.109 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.110 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:26.118 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:26.131 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:26.129+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:26.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2023-10-30T06:29:26.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:26.244 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:29:26.245 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:26.244+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:26.325 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.325 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:26.333 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:26.331+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:26.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:26.455 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:26.454+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:26.470 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.470 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:26.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:26.535 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:26.533+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:26.636 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:26.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:26.649 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.649 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:26.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:26.738 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:26.736+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:26.838 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:26.837+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:26.839 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:26.839 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:26.850 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.851 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:26.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:26.939 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:26.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:26.948 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:26.948 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:26.992 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:26.993 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:27.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:27.042 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:27.043 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.043 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:27.040+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:27.126 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:27.126 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.140 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:27.138+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:27.153 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:27.153 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:27.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:27.211 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:27.212 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2023-10-30T06:29:27.246 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:27.245+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:27.297 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:27.297 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.333 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:27.334 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:27.334 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:27.342 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:27.340+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:27.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:27.384 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.442 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:27.441+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:27.470 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:27.471 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:27.512 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:27.513 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:27.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:27.563 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:27.562+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:27.644 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:27.642+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:27.688 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:27.689 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:27.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:27.744 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:27.743+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:27.826 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:27.827 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:27.837 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:27.845 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:27.844+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:27.947 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:27.945+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:28.048 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:28.046+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 315.89417666600002, 2023-10-30T06:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 312.98100280300002, 2023-10-30T06:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 312.32082483400001, 2023-10-30T06:29:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 312.24540162699998, 2023-10-30T06:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.97427849600001, 2023-10-30T06:29:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.94399878600001, 2023-10-30T06:29:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.57008770200002, 2023-10-30T06:29:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.56558900099998, 2023-10-30T06:29:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.27398506200001, 2023-10-30T06:29:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.26804774499999, 2023-10-30T06:29:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.26674873000002, 2023-10-30T06:29:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 311.163759922, 2023-10-30T06:29:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.89185665500003, 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.88435846300001, 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.88369725799998, 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.87111141999998, 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.86501920699999, 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 310.86402386600003, 2023-10-30T06:29:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 303.54135129500003, 2023-10-30T06:29:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 303.50790981, 2023-10-30T06:29:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:28.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:29:28.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:28.292 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:28.288+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:28.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2023-10-30T06:29:28.415 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:28.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:28.461 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:28.462 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:28.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:28.517 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:28.516+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:28.593 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:28.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:28.629 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:28.630 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:28.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:28.693 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:28.692+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:28.795 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:28.793+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:28.818 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:28.818 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:28.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:28.818 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:28.819 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:28.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:28.912 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:28.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:28.996 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:28.995+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:29.000 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:29.000 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:29.000 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:29.001 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:29.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:29.077 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:00.958 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.081 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:00.962 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.097 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:29.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:29.109 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:00.989 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.153 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:20.367 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.155 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:20.369 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.175 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:29.175 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:29.176 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:29.176 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:29.176 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:29.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:29.192 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:20.407 110788== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:29.226 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:29.224+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:29.299 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:29.297+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:29.358 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:29.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:29.359 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 232.384018658, 2023-10-30T06:29:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:29.360 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.361 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:29.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:29.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:29.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:29.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:29.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:29.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:29.370 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:29.371 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 52.363526333999999, 2023-10-30T06:29:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:29.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.373 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:29.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:29.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:29.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:29.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:29.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:29.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:29.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:29.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:29.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:29.383 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:29.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:29.426 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:29.424+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:29.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2023-10-30T06:29:29.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:29.500 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:29.498+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:29.527 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:29.528 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:29.528 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:29.528 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:29.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:29.622 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:29.620+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:29.701 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:29.700+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:29.742 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:29.743 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:29.743 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:29.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:29.802 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:29.802 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:29.803 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:29.801+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:29.903 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:29.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:29.923 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:29.924 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:29.926 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:29.927 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:29.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:29.946 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:29:30.004 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:30.003+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:30.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:30.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.090 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.091 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:30.105 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:30.104+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:30.206 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:30.205+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:30.210 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:30.210 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.235 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.235 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:30.307 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:30.306+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:30.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:30.310 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.332 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:30.333 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:30.333 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2023-10-30T06:29:30.333 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:29:30.334 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:29:30.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2023-10-30T06:29:30.384 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.384 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:30.407 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:30.407 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.408 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:30.406+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:30.497 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:30.498 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2023-10-30T06:29:30.498 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:30.499 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:30.499 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.524 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:30.523+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:30.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2023-10-30T06:29:30.562 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.563 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:30.581 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:30.582 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:30.613 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:30.612+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:30.714 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:30.713+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:30.724 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.724 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:30.815 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:30.813+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:30.864 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:30.865 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:30.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:30.915 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:30.914+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:31.016 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:31.015+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:31.052 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.052 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:31.146 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:31.144+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:31.219 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:31.217+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:31.245 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.246 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:31.339 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:31.337+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:31.420 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.421 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.421 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:31.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:31.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:31.521 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:31.520+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:31.563 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.563 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.572 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:31.622 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:31.620+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:31.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2023-10-30T06:29:31.723 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:31.721+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:31.741 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.741 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:31.824 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:31.822+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:31.879 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:31.879 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:31.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:31.929 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:31.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:31.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:29:32.025 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:32.024+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:32.055 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.055 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:32.126 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:32.125+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:32.195 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.195 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:32.228 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:32.226+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:32.329 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:32.327+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:32.371 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.372 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:32.430 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:32.428+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:32.531 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:32.530+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:32.551 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.552 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.563 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:32.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:32.641 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:32.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:32.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2023-10-30T06:29:32.690 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.690 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:32.731 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:29:32.744 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:32.742+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:32.826 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.826 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:32.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:32.865 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:32.866 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:32.866 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:32.863+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:32.935 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:32.934+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:32.983 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:32.984 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:32.999 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:32.999 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:33.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:33.036 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:33.035+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:33.088 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:33.089 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.138 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:33.136+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:33.181 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:33.181 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:33.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:33.193 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:33.194 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.249 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:33.246+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:33.296 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:33.297 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.323 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:33.323 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:33.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:33.339 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:33.338+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:33.382 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.447 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:33.445+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:33.462 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:33.462 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:33.462 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:33.462 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:33.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:33.485 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:33.485 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.540 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:33.539+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:33.566 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:33.567 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:33.602 INFO:tasks.mon_recovery:adding mon b back in 2023-10-30T06:29:33.603 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-30T06:29:33.603 DEBUG:teuthology.orchestra.run.smithi112:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-10-30T06:29:33.606 INFO:tasks.ceph.mon.b:Started 2023-10-30T06:29:33.606 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:29:33.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:33.637 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:33.637 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:33.637 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:33.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:33.650 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:33.648+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:33.742 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:33.741+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:33.824 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:33.824 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:33.825 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:33.825 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:33.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:33.878 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:33.876+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:33.943 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:33.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:33.962 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:33.962 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:33.962 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:33.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:34.057 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:34.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:34.144 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:34.144 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:34.145 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:34.145 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:34.145 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:34.145 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:34.143+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:34.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:34.245 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:34.244+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:34.346 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:34.345+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:34.376 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:34.378 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:34.378 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:34.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:34.378 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:34.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.379 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:34.379 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:34.379 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 321.04022450999997, 2023-10-30T06:29:34.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:34.380 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.380 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.380 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:34.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:34.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:34.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:34.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:34.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:34.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:34.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:34.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:34.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:34.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:34.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:34.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:34.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:34.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:34.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:34.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:34.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:34.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:34.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:34.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:34.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:34.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:34.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:34.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:34.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:34.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:34.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.397 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.397 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.398 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.398 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:34.398 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.71906931900003, 2023-10-30T06:29:34.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:34.399 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.399 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.399 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.400 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.400 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:34.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.400 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:34.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:34.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:34.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:34.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:34.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:34.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:34.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:34.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:34.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:34.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:34.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:34.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:34.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:34.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:34.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:34.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:34.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:34.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:34.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:34.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:34.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:34.415 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.71482476199998, 2023-10-30T06:29:34.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:34.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.416 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.416 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:34.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.417 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:34.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:34.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:34.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:34.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:34.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:34.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:34.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:34.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:34.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:34.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:34.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:34.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:34.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:34.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:34.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:34.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:34.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:34.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:34.429 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.430 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.430 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.430 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.431 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:34.431 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.62936633700002, 2023-10-30T06:29:34.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:34.431 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.431 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.432 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.432 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:34.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.433 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:34.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:34.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:34.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:34.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:34.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:34.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:34.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:34.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:34.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:34.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:34.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:34.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:34.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:34.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:34.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:34.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:34.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.446 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.447 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:34.447 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.36012312100002, 2023-10-30T06:29:34.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:34.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.448 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:34.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:34.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:34.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:34.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:34.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:34.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:34.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:34.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:34.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:34.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:34.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:34.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:34.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:34.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:34.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:34.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:34.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.461 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.461 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.462 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.462 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:34.462 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.35929170999998, 2023-10-30T06:29:34.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.463 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:34.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:34.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:34.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:34.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:34.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:34.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:34.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:34.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:34.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:34.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:34.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:34.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:34.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:34.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:34.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:34.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:34.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.477 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.477 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:34.478 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.35862458299999, 2023-10-30T06:29:34.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:34.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.478 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.478 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.479 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.479 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:34.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.479 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:34.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:34.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:34.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:34.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:34.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:34.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:34.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:34.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:34.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:34.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:34.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:34.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:34.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:34.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:34.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:34.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:34.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.493 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.493 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:34.493 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.33635051200002, 2023-10-30T06:29:34.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:34.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.494 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.494 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.495 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.495 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:34.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.495 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:34.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:34.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:34.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:34.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:34.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:34.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:34.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:34.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:34.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:34.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:34.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:34.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:34.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:34.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:34.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:34.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:34.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:34.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:34.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:34.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.509 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.509 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:34.509 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.33570265600002, 2023-10-30T06:29:34.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:34.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.510 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.510 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:34.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:34.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:34.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:34.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:34.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:34.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:34.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:34.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:34.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:34.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:34.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:34.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:34.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:34.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:34.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:34.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:34.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:34.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:34.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 318.33501658300003, 2023-10-30T06:29:34.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:34.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:34.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.527 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:34.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:34.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:34.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:34.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:34.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:34.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:34.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:34.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:34.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:34.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:34.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:34.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:34.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:34.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:34.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:34.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:34.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:34.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:34.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:34.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:34.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.26413461499999, 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:34.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:34.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:34.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:34.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:34.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:34.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:34.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:34.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:34.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:34.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:34.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:34.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:34.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:34.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:34.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:34.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:34.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:34.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:34.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:34.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:34.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:34.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:34.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.263470649, 2023-10-30T06:29:34.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:34.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:34.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:34.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:34.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:34.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:34.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:34.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:34.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:34.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:34.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:34.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:34.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:34.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:34.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:34.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:34.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:34.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:34.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:34.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:34.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:34.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.573 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:34.573 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:34.573 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 317.23501684000001, 2023-10-30T06:29:34.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:34.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.574 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.574 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:34.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:34.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:34.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:34.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:34.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:34.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:34.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:34.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:34.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:34.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:34.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:34.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:34.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:34.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:34.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:34.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:34.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:34.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:34.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:34.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.89325034699999, 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:34.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.591 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:34.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:34.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:34.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:34.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:34.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:34.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:34.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:34.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:34.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:34.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:34.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:34.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:34.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:34.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:34.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:34.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:34.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:34.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:34.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.603 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.604 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.604 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.604 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:34.605 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.89250167699998, 2023-10-30T06:29:34.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:34.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.605 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.605 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:34.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:34.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:34.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:34.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:34.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:34.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:34.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:34.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:34.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:34.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:34.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:34.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:34.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:34.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:34.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:34.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:34.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.620 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.620 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:34.621 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.89183329399998, 2023-10-30T06:29:34.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:34.621 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.621 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.621 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.622 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.622 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:34.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.622 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:34.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:34.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:34.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:34.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:34.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:34.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:34.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:34.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:34.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:34.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:34.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:34.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:34.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:34.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:34.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:34.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.636 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.636 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:34.636 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.88198261999997, 2023-10-30T06:29:34.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:34.637 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.637 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.637 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.638 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:34.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.638 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:34.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:34.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:34.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:34.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:34.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:34.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:34.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:34.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:34.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:34.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:34.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:34.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:34.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:34.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:34.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:34.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:34.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:34.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:34.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:34.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.651 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.652 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.652 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:34.652 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.881308805, 2023-10-30T06:29:34.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:34.653 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.653 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.653 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.654 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.654 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:34.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.654 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:34.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:34.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:34.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:34.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:34.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:34.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:34.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:34.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:34.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:34.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:34.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:34.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:34.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:34.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:34.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:34.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:34.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:34.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.668 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:34.668 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.88066042200001, 2023-10-30T06:29:34.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:34.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:34.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.670 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:34.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:34.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:34.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:34.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:34.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:34.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:34.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:34.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:34.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:34.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:34.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:34.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:34.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:34.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:34.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:34.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:34.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:34.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.682 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.682 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:34.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 309.87654395599998, 2023-10-30T06:29:34.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:34.684 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:34.684 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:34.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:34.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:34.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:34.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:34.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:34.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:34.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:34.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:34.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:34.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:34.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:34.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:34.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:34.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:34.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:34.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:34.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:34.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:34.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:34.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:34.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:34.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:34.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:34.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:34.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:34.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:34.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:34.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:34.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:34.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:34.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.698 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:34.698 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:34.699 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:34.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:34.702 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:34.699+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:34.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:29:34.801 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:34.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:34.872 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:34.873 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:34.873 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:34.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:34.902 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:34.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:35.003 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:35.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:35.012 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:35.013 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:35.014 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:35.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:35.104 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:35.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:35.151 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:35.151 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:35.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:35.151 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:35.151 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:35.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:35.206 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:35.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:35.290 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:35.290 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:35.290 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:35.291 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:35.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:35.306 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:35.304+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:35.406 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:35.405+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:35.467 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:35.467 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:35.467 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:35.467 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:35.468 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:35.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:35.507 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:35.507 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:35.507 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 7, 6] 2023-10-30T06:29:35.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2023-10-30T06:29:35.508 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:35.506+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:35.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:35.630 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:35.631 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:35.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:35.631 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:35.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.632 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:35.632 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:35.632 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 239.333896692, 2023-10-30T06:29:35.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:35.632 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:35.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:35.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:35.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:35.633 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:35.633 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:35.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.634 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:35.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:35.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:35.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:35.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:35.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:35.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:35.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:35.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:35.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:35.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:35.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:35.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:35.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:35.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:35.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:35.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:35.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:35.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:35.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:35.642 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:35.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:35.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:35.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:35.643 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 59.301722472999998, 2023-10-30T06:29:35.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:35.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:35.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:35.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:35.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:35.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:35.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:35.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:35.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:35.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:35.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:35.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:35.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:35.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:35.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:35.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:35.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:35.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:35.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:35.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:35.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:35.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:35.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:35.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:35.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:35.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:35.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:35.653 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:35.654 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:35.630+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:35.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:35.709 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:35.708+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:35.785 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:35.786 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:35.786 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:35.786 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:35.787 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:29:35.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:35.810 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:35.809+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:35.914 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:35.914 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:35.922 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:35.913+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:35.964 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:35.964 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:35.965 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:35.965 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:35.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:36.013 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:36.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:36.024 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:36.024 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.103 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:36.104 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:36.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:36.104 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:36.104 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:36.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:36.114 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:36.113+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:36.128 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:36.128 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.215 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:36.214+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:36.220 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:36.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.240 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:29:36.226+0000 ff6f640 -1 mon.a@0(leader).osd e93 definitely_dead 0 2023-10-30T06:29:36.246 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:36.247 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:36.247 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:36.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:36.308 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:36.308 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.340 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:36.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:36.408 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.429 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:36.427+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:36.443 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:36.443 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:36.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:36.517 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:36.516+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:36.535 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:36.535 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.618 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:36.617+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:36.621 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:36.622 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:36.622 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:36.623 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:36.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:36.720 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:36.718+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:36.761 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:36.761 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:36.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:36.820 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:36.818+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:36.898 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:36.899 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:36.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:36.921 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:36.919+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:36.956 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.3. 2023-10-30T06:29:37.021 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:37.020+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:37.036 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.036 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:37.122 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:37.121+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:37.212 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.212 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:37.222 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:37.221+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:37.324 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:37.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:37.432 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:37.430+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:37.470 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.470 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:37.563 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:37.562+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:37.629 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:37.626+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:37.649 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.650 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:37.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:37.729 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:37.727+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:37.789 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.789 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:37.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:29:37.844 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:37.842+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:37.931 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:37.929+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:37.949 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:37.949 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:37.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:38.031 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:38.030+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:38.132 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:38.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:38.153 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:38.153 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:38.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:38.249 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:38.247+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:38.334 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:38.332+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:38.357 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:38.357 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:38.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:38.423 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":39,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:38.434 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:38.433+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:38.535 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:38.534+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:38.556 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:38.556 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:38.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:38.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:38.636 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:38.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:38.694 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:38.694 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:38.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:38.737 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:38.736+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:38.839 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:38.837+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:38.892 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:38.892 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:38.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:38.895 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:38.896 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:38.940 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:38.938+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:39.038 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:29:39.040 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:39.038+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:39.047 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:39.048 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.092 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:39.092 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:39.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:39.141 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:39.140+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:39.168 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:39.168 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.242 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:39.241+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:39.286 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:39.286 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.298 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:39.299 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:39.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:39.299 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:39.300 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:39.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:39.350 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:39.349+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:39.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:39.373 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.444 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:39.442+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:39.462 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:39.462 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.481 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:39.482 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:39.482 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:39.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:39.544 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:39.543+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:39.550 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:39.551 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.645 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:39.644+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:39.651 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:39.651 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:39.672 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:39.672 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:39.672 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:39.672 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:39.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:39.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:39.765 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:39.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:39.847 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:39.846+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:39.852 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:39.853 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:39.854 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 243.55936249600001, 2023-10-30T06:29:39.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:39.854 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:39.854 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:39.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:39.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:39.855 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:39.855 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:39.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.856 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:39.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:39.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:39.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:39.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:39.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:39.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:39.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:39.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:39.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:39.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:39.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:39.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:39.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:39.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:39.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:39.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:39.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:39.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:39.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:39.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.865 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:39.865 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:39.865 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 63.527188277, 2023-10-30T06:29:39.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:39.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:39.866 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:39.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:39.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:39.866 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:39.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:39.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:39.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:39.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:39.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:39.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:39.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:39.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:39.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:39.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:39.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:39.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:39.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:39.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:39.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:39.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:39.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:39.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:39.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:39.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:39.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:39.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:39.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:39.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:39.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:39.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:39.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:39.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:39.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:39.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:39.876 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:39.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:39.948 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:39.947+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:40.023 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:40.024 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:40.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:40.049 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:40.047+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:40.151 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:40.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:40.230 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.230 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:40.230 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:40.231 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:40.231 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:40.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:40.286 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:40.284+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:40.353 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:40.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:40.369 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:40.370 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:40.370 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:40.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:40.463 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:40.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:40.548 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:40.549 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:40.549 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:40.554 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:40.552+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:40.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:40.656 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:40.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:40.731 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.732 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:40.732 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:40.732 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:40.733 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:40.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:40.783 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:40.782+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:40.858 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:40.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:40.954 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:40.954 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:40.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:40.955 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 328.79800121300002, 2023-10-30T06:29:40.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:40.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:40.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:40.956 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:40.956 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:40.957 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:40.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:40.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.957 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:40.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:40.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:40.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:40.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:40.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:40.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:40.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:40.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:40.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:40.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:40.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:40.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:40.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:40.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:40.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:40.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:40.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:40.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:40.967 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.968 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:40.968 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:40.968 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 325.88482735000002, 2023-10-30T06:29:40.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:40.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:40.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:40.969 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:40.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.970 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:40.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:40.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:40.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:40.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:40.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:40.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:40.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:40.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:40.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:40.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:40.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:40.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:40.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:40.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:40.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:40.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:40.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.980 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:40.980 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:40.980 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 325.22464938100001, 2023-10-30T06:29:40.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:40.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:40.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:40.981 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:40.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:40.982 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:40.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:40.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:40.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:40.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:40.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:40.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:40.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:40.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:40.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:40.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:40.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:40.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:40.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:40.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:40.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:40.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:40.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:40.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:40.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:40.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:40.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:40.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 325.14922617399998, 2023-10-30T06:29:40.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:40.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.004 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:41.004 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:41.004 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.87810304300001, 2023-10-30T06:29:41.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:41.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.005 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:41.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:41.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:41.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:41.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:41.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:41.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:41.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:41.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:41.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:41.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:41.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:41.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:41.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:41.015 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.015 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.016 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.016 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:41.016 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.84782333300001, 2023-10-30T06:29:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.017 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:41.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:41.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:41.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:41.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:41.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:41.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:41.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:41.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:41.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:41.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:41.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:41.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:41.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:41.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:41.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:41.027 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.027 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:41.028 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.47391224900002, 2023-10-30T06:29:41.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:41.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.029 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:41.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:41.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:41.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:41.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:41.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:41.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:41.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:41.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:41.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:41.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:41.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:41.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:41.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:41.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:41.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:41.039 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.039 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.040 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:41.040 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:41.041 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.46941354799998, 2023-10-30T06:29:41.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:41.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.041 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:41.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:41.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:41.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:41.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:41.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:41.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:41.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:41.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:41.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:41.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:41.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:41.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:41.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:41.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.051 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.052 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.052 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:41.052 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:41.053 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.17780960900001, 2023-10-30T06:29:41.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:41.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.053 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:41.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:41.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:41.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:41.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:41.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:41.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:41.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:41.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:41.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:41.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:41.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:41.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:41.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:41.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:41.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:41.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:41.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:41.063 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.064 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:41.065 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:41.065 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.17187229199999, 2023-10-30T06:29:41.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:41.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.066 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:41.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:41.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:41.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:41.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:41.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:41.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:41.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:41.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:41.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:41.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:41.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:41.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:41.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:41.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:41.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:41.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.076 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.077 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:41.077 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:41.077 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.17057327700002, 2023-10-30T06:29:41.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:41.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.078 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:41.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:41.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:41.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.082 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:41.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:41.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:41.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:41.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:41.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:41.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:41.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:41.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:41.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:41.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:41.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:41.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.088 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 324.067584469, 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:41.089 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.090 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:41.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:41.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:41.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:41.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:41.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:41.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:41.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:41.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:41.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:41.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:41.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:41.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:41.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:41.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:41.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:41.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.100 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.101 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:41.101 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:41.101 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.79568120200003, 2023-10-30T06:29:41.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:41.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.102 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:41.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:41.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:41.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:41.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:41.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:41.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:41.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:41.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:41.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:41.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:41.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:41.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:41.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:41.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:41.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:41.112 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.112 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:41.113 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.78818301000001, 2023-10-30T06:29:41.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:41.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.114 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:41.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:41.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:41.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:41.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:41.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:41.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:41.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:41.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:41.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:41.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:41.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:41.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:41.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:41.124 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.124 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.124 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.125 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:41.125 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:41.125 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.78752180499998, 2023-10-30T06:29:41.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:41.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.126 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:41.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:41.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:41.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:41.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:41.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:41.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:41.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:41.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:41.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:41.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:41.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:41.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:41.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:41.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.136 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.137 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:41.137 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:41.137 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.77493596699998, 2023-10-30T06:29:41.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:41.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.138 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:41.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:41.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:41.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:41.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:41.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:41.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:41.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:41.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:41.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:41.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:41.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:41.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:41.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:41.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:41.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.150 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.150 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.151 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:41.151 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:41.151 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.76884375399999, 2023-10-30T06:29:41.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:41.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.152 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:41.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:41.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:41.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:41.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:41.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:41.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:41.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:41.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:41.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:41.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:41.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:41.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:41.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:41.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:41.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:41.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.162 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.163 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:41.163 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:41.163 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 323.76784841300002, 2023-10-30T06:29:41.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:41.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:41.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:41.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:41.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:41.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:41.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:41.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:41.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:41.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.174 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.175 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:41.175 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:41.175 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 316.44517584200003, 2023-10-30T06:29:41.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:41.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.176 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:41.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:41.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:41.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:41.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:41.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:41.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:41.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:41.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:41.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:41.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:41.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:41.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:41.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:41.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:41.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:41.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.186 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 316.411734357, 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:41.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:41.188 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:41.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:41.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:41.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:41.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:41.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:41.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:41.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:41.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:41.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:41.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:41.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:41.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:41.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:41.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:41.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:41.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:41.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:41.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:41.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:41.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:41.197 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.197 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:41.198 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:41.198 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:41.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:29:41.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:41.201 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:41.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:41.314 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:41.312+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:41.328 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:41.328 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:41.328 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:41.329 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:41.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:41.422 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:41.420+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:41.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:41.503 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:41.501+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:41.514 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:41.515 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:41.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:41.604 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:41.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:41.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:41.689 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:41.691 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:41.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:41.743 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:41.742+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:41.798 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.133 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:41.806 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:41.804+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:41.827 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:41.827 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:41.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:41.919 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:41.919 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:41.920 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:41.919+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:41.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:41.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:42.002 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:42.003 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:42.007 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:42.006+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:42.009 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:42.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.012 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:42.032 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:08.367 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:42.098 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.109 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:42.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:42.167 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 245.190150601, 2023-10-30T06:29:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 65.169658276999996, 2023-10-30T06:29:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:42.192 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:42.193 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:42.194 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:42.203 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":43,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:42.214 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:42.213+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:42.223 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:29:42.279 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:42.280 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.309 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:42.308+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:42.353 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:42.353 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:42.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:42.367 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:42.368 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.411 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:42.410+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:42.455 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:42.456 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.496 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:42.496 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:42.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:42.541 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:42.541 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:42.542 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:42.539+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:42.612 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:42.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:42.668 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:42.668 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:42.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:42.713 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:42.712+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:42.814 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:42.813+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:42.843 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:42.844 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:42.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:42.916 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:42.914+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:42.982 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:42.983 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:42.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:43.033 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:43.032+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:43.118 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:43.117+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:43.165 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:43.166 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:43.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:43.219 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:43.217+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:43.319 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:43.319 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:43.319 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:43.318+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:43.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:43.420 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:43.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:43.495 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:43.495 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:43.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:43.521 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:43.520+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:43.623 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:43.621+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:43.672 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:43.673 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:43.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:43.724 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:43.722+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:43.824 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:43.823+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:43.849 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:43.849 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:43.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:43.925 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:43.924+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:44.027 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:44.025+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:44.036 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.036 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:44.127 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:44.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:44.225 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.225 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.227 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:44.226+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:44.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:44.242 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:29:44.329 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:44.327+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:44.366 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.366 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:44.429 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:44.428+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:44.505 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.506 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:44.530 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:44.528+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:44.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:44.631 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:44.630+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:44.688 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.688 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:44.741 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:44.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:44.813 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:44.833 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:44.832+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:44.837 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:44.837 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:44.847 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:44.911 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:44.912 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:44.940 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:44.939+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:45.009 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:45.010 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.014 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:45.015 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:45.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:45.068 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:45.066+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:45.092 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:45.093 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.156 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:45.157 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:45.165 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:45.182 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:45.181+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:45.183 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:45.237 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:45.236+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:45.289 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:45.290 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.338 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:45.336+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:45.379 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:45.380 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:45.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:45.402 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:11.736 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:45.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:45.429 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.455 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:45.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:45.462 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:11.797 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:45.531 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:45.533 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:45.533 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:45.533 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:45.533 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:45.533 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:45.539 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:45.537+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:45.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:45.640 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:45.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:45.711 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:45.712 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:45.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:45.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:45.712 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:45.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:45.725 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:12.060 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:29:45.740 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:45.739+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:45.757 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:29:45.841 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:45.839+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:45.884 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:45.885 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:45.885 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:45.885 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:45.889 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":46,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:45.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:45.944 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:45.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:46.025 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:46.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:46.026 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:46.026 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:46.026 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:46.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:46.042 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:46.041+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:46.143 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:46.142+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:46.173 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:46.173 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:46.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:46.174 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:46.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.174 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:46.174 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:46.175 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 249.19818258999999, 2023-10-30T06:29:46.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:46.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:46.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:46.176 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:46.176 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:46.176 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:46.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:46.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.177 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:46.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:46.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:46.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:46.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:46.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:46.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:46.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:46.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:46.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:46.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:46.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:46.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:46.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:46.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:46.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:46.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.184 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:46.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:46.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:46.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:46.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:46.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:46.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:46.186 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:46.186 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:46.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.187 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:46.187 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:46.188 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 69.177690265999999, 2023-10-30T06:29:46.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:46.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:46.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:46.189 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:46.189 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:46.189 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:46.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:46.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:46.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:46.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:46.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:46.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:46.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:46.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:46.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:46.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:46.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:46.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:46.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:46.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:46.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:46.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:46.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:46.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:46.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:46.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:46.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:46.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:46.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:46.198 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:46.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:46.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:46.199 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:46.199 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:46.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:46.244 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:46.243+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:46.328 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:46.328 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:46.329 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:46.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:46.345 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:46.344+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:46.446 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:46.446+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:46.503 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:46.503 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:46.504 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:46.504 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:46.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:46.555 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:46.554+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:46.639 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:46.639 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:46.640 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:46.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:46.648 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:46.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:46.749 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:46.748+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:46.817 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:46.818 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:46.818 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:46.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:46.871 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:46.869+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:46.951 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:46.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:46.958 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:46.958 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:46.959 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:46.959 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:46.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:46.971 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:46.971 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:46.971 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2023-10-30T06:29:46.971 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:29:46.972 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:29:46.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2023-10-30T06:29:47.053 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:47.051+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:47.153 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:47.151+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:47.158 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:47.159 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2023-10-30T06:29:47.159 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:47.218 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 333.87657209700001, 2023-10-30T06:29:47.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.220 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.221 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:47.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:47.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:47.226 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:47.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:47.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.229 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:47.230 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.231 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:47.238 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.239 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.555416906, 2023-10-30T06:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.240 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:47.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:47.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.247 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.248 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.255 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.256 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.55117234900001, 2023-10-30T06:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.257 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.258 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:47.264 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:47.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:47.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:47.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:47.273 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.274 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.465713924, 2023-10-30T06:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.276 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.277 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:47.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:47.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:47.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:47.283 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:47.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:47.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:47.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:47.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:47.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:47.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:47.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:47.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:47.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:47.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:47.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:47.291 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.291 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.292 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.292 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.292 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:47.293 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.19647070799999, 2023-10-30T06:29:47.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:47.293 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.293 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.294 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.294 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:47.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.295 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:47.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:47.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:47.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:47.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:47.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:47.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:47.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:47.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:47.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:47.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:47.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:47.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:47.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:47.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:47.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:47.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:47.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:47.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:47.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:47.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:47.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:47.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:47.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:47.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.309 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.309 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.310 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.310 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.310 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:47.310 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.19563929700001, 2023-10-30T06:29:47.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:47.311 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.311 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.311 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:47.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:47.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:47.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:47.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:47.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:47.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:47.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:47.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:47.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:47.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:47.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:47.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:47.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:47.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:47.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:47.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:47.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.19497217000003, 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.328 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.329 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.329 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:47.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:47.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:47.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:47.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:47.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:47.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:47.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:47.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:47.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:47.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:47.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:47.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:47.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:47.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:47.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:47.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:47.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:47.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:47.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:47.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:47.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:47.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:47.343 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.343 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.344 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.344 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.344 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:47.345 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.172698099, 2023-10-30T06:29:47.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:47.345 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.345 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.345 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.346 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.346 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:47.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.346 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:47.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:47.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:47.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:47.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:47.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:47.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:47.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:47.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:47.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:47.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:47.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:47.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:47.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:47.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:47.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:47.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:47.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:47.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:47.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:47.359 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.359 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.360 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.360 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.360 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:47.361 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.172050243, 2023-10-30T06:29:47.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:47.361 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.361 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.361 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.362 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.362 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:47.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.362 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:47.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:47.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:47.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:47.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:47.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:47.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:47.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:47.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:47.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:47.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:47.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:47.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:47.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:47.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:47.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:47.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:47.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:47.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:47.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:47.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.376 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.376 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.377 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:47.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:47.378 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 331.17136417, 2023-10-30T06:29:47.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:47.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:47.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.379 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:47.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:47.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:47.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:47.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:47.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:47.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:47.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:47.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:47.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:47.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:47.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:47.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:47.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:47.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:47.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:47.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:47.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:47.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:47.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:47.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:47.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:47.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.394 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:47.394 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:47.394 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 330.10048220200002, 2023-10-30T06:29:47.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:47.394 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.395 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:47.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:47.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:47.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:47.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:47.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:47.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:47.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:47.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:47.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:47.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:47.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:47.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:47.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:47.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:47.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:47.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:47.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:47.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:47.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:47.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:47.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.409 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.409 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.410 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:47.410 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:47.410 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 330.09981823599998, 2023-10-30T06:29:47.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.411 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:47.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.412 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:47.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:47.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:47.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:47.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:47.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:47.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:47.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:47.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:47.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:47.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:47.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:47.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:47.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:47.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:47.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:47.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:47.425 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.426 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.426 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 330.07136442699999, 2023-10-30T06:29:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:47.427 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:47.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:47.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:47.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:47.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:47.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:47.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:47.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:47.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:47.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:47.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:47.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.443 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.443 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:47.443 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.72959793400003, 2023-10-30T06:29:47.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:47.443 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.444 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:47.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.445 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:47.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:47.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:47.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:47.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:47.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:47.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:47.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:47.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:47.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:47.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:47.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:47.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:47.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:47.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:47.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:47.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:47.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:47.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:47.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:47.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:47.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:47.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:47.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.458 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.458 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.459 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.459 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:47.459 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.72884926400002, 2023-10-30T06:29:47.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:47.460 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.460 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.460 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.461 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.461 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:47.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.461 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:47.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:47.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:47.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:47.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:47.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:47.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:47.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:47.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:47.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:47.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:47.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:47.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:47.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:47.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:47.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:47.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:47.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:47.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:47.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:47.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.474 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.474 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.475 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.475 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:47.475 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.72818088100001, 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.476 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.477 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:47.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.477 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:47.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:47.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:47.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:47.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:47.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:47.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:47.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:47.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:47.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:47.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:47.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:47.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:47.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:47.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:47.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:47.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:47.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:47.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:47.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:47.490 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.490 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.491 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.491 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.491 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:47.492 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.71833020700001, 2023-10-30T06:29:47.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:47.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.492 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.492 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:47.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:47.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:47.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:47.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:47.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:47.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:47.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:47.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:47.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:47.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:47.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:47.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:47.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:47.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:47.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.71765639199998, 2023-10-30T06:29:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:47.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.509 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.523 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.524 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.71700800899998, 2023-10-30T06:29:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.525 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.526 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:47.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:47.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:47.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:47.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:47.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:47.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:47.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:47.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:47.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:47.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:47.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 322.71289154300001, 2023-10-30T06:29:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:47.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:47.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:47.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:47.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:47.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:47.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:47.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:47.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:47.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:47.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:47.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:47.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:47.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:47.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:47.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:47.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:47.556 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:47.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:47.558 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:47.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:47.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:47.659 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:47.657+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:47.732 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:47.733 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:47.733 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:47.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:47.759 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:47.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:47.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:29:47.840 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:47.840 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:47.867 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:47.865+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:47.919 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:47.920 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:47.921 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:47.921 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:47.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:47.945 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:47.945 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:47.989 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:47.987+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:48.036 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:48.037 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.063 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:48.062+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:48.080 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.080 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:48.152 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:48.152 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.163 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:48.163+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:48.240 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:48.240 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.265 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.265 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.267 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:48.267+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:48.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:48.325 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:48.325 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.365 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:48.364+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:48.415 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:48.415 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.444 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.444 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:48.494 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:48.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:48.498 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:48.498 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:48.567 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:48.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:48.621 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.621 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:48.671 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:48.670+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:48.757 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.758 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:48.768 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:48.766+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:48.869 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:48.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:48.890 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:48.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:48.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:48.970 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:48.969+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:49.028 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.028 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:49.071 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:49.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:49.107 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:29:49.172 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:49.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:49.204 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.205 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:49.273 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:49.272+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:49.340 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.341 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:49.375 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:49.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:49.476 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:49.474+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:49.516 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.517 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:49.577 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:49.575+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:49.630 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":50,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:49.685 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:49.684+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:49.692 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.692 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:49.779 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:49.777+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:49.827 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.827 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:49.880 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:49.879+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:49.981 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:49.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:49.989 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:49.989 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:49.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:50.082 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:50.081+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:50.124 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:50.125 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:50.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:50.183 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:50.182+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:50.258 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:50.258 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:50.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:50.284 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:50.283+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:50.387 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:50.384+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:50.393 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:50.393 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:50.402 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:50.488 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:50.486+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:50.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:50.532 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:50.533 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:50.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:50.628 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:50.627+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:50.690 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:50.690+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:50.758 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:50.759 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:50.760 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:50.760 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:50.760 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:50.760 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:50.760 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:50.769 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:50.791 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:50.789+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:50.844 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:50.845 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:50.892 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:50.891+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:50.976 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:50.976 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:50.978 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:50.978 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:50.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:50.978 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:50.979 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:50.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:51.030 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:51.029+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:51.074 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:51.075 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:51.093 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:51.092+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:51.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:29:51.157 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:51.157 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:51.158 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:51.158 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:51.158 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:51.158 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:51.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:51.194 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:51.192+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:51.252 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:51.252 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:51.336 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:51.334+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:51.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:51.337 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:51.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:51.348 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:51.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:51.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:51.395 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:51.394+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:51.420 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:51.421 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:51.496 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:51.495+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:51.543 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:51.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:29:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:51.544 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 338.20431664300003, 2023-10-30T06:29:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:29:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.545 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:29:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:29:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:29:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:29:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:29:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:29:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:29:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:29:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:29:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:29:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:29:51.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:29:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:29:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:29:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:29:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:29:51.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:29:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:29:51.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:29:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:29:51.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:29:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:29:51.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:29:51.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.561 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.88316145200002, 2023-10-30T06:29:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:29:51.562 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:29:51.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:51.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:29:51.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:29:51.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:29:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:29:51.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:29:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:29:51.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:29:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:29:51.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:29:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:29:51.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:29:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:29:51.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:29:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:29:51.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:29:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:29:51.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:29:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:29:51.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:29:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:29:51.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:29:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:29:51.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:29:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.87891689499997, 2023-10-30T06:29:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:29:51.578 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.579 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:29:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:29:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:29:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:29:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:29:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:29:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:29:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:29:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:29:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:29:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:29:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:29:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:29:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:29:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:29:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:29:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:29:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:29:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:29:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:29:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.79345847000002, 2023-10-30T06:29:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:29:51.594 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.595 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:29:51.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:29:51.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:29:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:29:51.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:29:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:29:51.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:29:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:29:51.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:29:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:29:51.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:29:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:29:51.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:29:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:29:51.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:29:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:29:51.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:29:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:29:51.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:29:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:29:51.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:29:51.608 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.609 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.52421525400001, 2023-10-30T06:29:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:29:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.610 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:29:51.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:29:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:29:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:29:51.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:29:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:29:51.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:29:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:29:51.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:29:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:29:51.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:29:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:29:51.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:29:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:29:51.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:29:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:29:51.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:29:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:29:51.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:29:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:29:51.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:29:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:29:51.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:29:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:29:51.624 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.625 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.52338384299998, 2023-10-30T06:29:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:29:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.626 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:29:51.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:29:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:29:51.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:29:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:29:51.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:29:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:29:51.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:29:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:29:51.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:29:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:29:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:29:51.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:29:51.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:29:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:29:51.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:29:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:29:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:29:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:29:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:29:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:29:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:29:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.52271671599999, 2023-10-30T06:29:51.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:29:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.643 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:29:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:29:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:29:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:29:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:29:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:29:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:29:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:29:51.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:29:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:29:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:29:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:29:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:29:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:29:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:29:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:29:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:29:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:29:51.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:29:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.50044264500002, 2023-10-30T06:29:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:29:51.658 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.659 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:51.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:29:51.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:29:51.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:29:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:29:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:29:51.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:29:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:29:51.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:29:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:29:51.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:29:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:29:51.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:29:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:29:51.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:29:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:29:51.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:29:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:29:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.673 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.49979478900002, 2023-10-30T06:29:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:29:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.674 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:29:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.675 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:29:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:29:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:29:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:29:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:29:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:29:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:29:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:29:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:29:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:29:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:29:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:29:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:29:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:29:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:29:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:29:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:29:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:29:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.49910871600002, 2023-10-30T06:29:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:29:51.689 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.690 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:29:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:29:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:29:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:29:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:29:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:29:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:29:51.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:29:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:29:51.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:29:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:29:51.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:29:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:29:51.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:29:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:29:51.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:29:51.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:29:51.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:29:51.703 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.704 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 334.42822674799999, 2023-10-30T06:29:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:29:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.705 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:29:51.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:51.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:29:51.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:29:51.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:29:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:29:51.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:29:51.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:29:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:29:51.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:29:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:29:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:29:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:29:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:29:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:29:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:29:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:29:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:29:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:29:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:51.720 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 334.427562782, 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:29:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:29:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:29:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:29:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:29:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:29:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:29:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:29:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:29:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:29:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:29:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:29:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:29:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:29:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:29:51.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:29:51.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:29:51.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:29:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 334.39910897300001, 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.736 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:29:51.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:51.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:29:51.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:29:51.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:29:51.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:29:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:29:51.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:29:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:29:51.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:29:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:29:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:29:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:29:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:29:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:29:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:29:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:29:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:29:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:29:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.750 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.05734247999999, 2023-10-30T06:29:51.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:29:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.752 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:29:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:29:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:29:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:29:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:29:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:29:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:29:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:29:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:29:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:29:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:29:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:29:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:29:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:29:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:29:51.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:29:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:29:51.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:29:51.765 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.766 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.05659380999998, 2023-10-30T06:29:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:29:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.767 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:29:51.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:29:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:29:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:29:51.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:29:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:29:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:29:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:29:51.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:29:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:29:51.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:29:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:29:51.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:29:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:29:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:29:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:29:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:29:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:29:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:29:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:29:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.781 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:51.782 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.05592542699998, 2023-10-30T06:29:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:29:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.783 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:29:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:29:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:29:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:29:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:29:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:29:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:29:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:29:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:29:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:29:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:29:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:29:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:29:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:29:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:29:51.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:29:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:29:51.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:29:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.04607475300003, 2023-10-30T06:29:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:29:51.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.799 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:29:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.801 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:29:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.802 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:29:51.803 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:29:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:29:51.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:29:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:29:51.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:29:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:29:51.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:29:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:29:51.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:29:51.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:29:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:29:51.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:29:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:29:51.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:29:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:29:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.813 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.045400938, 2023-10-30T06:29:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:29:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.814 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:29:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.815 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:29:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:29:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:29:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:29:51.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:29:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:29:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:29:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:29:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:29:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:29:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:29:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:29:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:29:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:29:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:29:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:29:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:29:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:29:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:29:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:29:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:51.831 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.044752555, 2023-10-30T06:29:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:29:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.832 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.833 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:29:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:29:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:29:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:29:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:29:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:29:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:29:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:29:51.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:29:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:29:51.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:29:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:29:51.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:29:51.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:29:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:29:51.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:29:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:29:51.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:29:51.846 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 327.04063608899997, 2023-10-30T06:29:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:29:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:51.847 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:29:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:29:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.848 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:29:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:29:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:29:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:51.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:29:51.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:29:51.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:51.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:29:51.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:29:51.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:29:51.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:29:51.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:29:51.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:29:51.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:29:51.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:29:51.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:29:51.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:29:51.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:29:51.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:29:51.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:29:51.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:29:51.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:51.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:51.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:51.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:29:51.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:51.861 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:51.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:51.864 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:51.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:51.981 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:51.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:51.997 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:51.997 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:51.998 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:51.998 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:52.013 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:29:52.065 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:52.064+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:52.156 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.157 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:52.157 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:52.157 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:52.157 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:52.166 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:52.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:52.170 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:52.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:52.170 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2023-10-30T06:29:52.170 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:29:52.171 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:29:52.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2023-10-30T06:29:52.266 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:52.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:52.301 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.302 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:52.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:52.302 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:52.302 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:52.341 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.342 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "" 2023-10-30T06:29:52.342 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.367 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:52.366+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:52.437 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.438 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:52.438 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:52.438 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:52.468 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:52.466+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:52.569 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:52.568+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:52.571 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:52.571 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:52.572 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:52.572 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:52.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:52.708 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:52.707+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:52.753 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:52.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:29:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:29:52.754 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 256.45963477200002, 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:52.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:29:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:52.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:29:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:29:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:29:52.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:29:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:29:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:52.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 76.427460553000003, 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:29:52.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:29:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:29:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:29:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:29:52.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:52.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:29:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:29:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:29:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:29:52.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:29:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:29:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:29:52.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:29:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:29:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:29:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:29:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:29:52.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:29:52.774 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:29:52.775 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:52.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:52.777 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:52.776+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:52.877 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:52.876+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:52.910 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:52.910 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:52.910 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:52.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:52.978 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:52.978+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:53.079 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:53.079+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:53.085 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:53.085 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:53.085 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:53.086 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:53.086 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:53.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:53.180 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:53.179+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:53.228 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:53.229 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:53.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:53.229 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:53.229 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:53.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:53.281 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:53.280+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:53.291 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":54,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:53.382 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:53.382+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:53.402 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:53.403 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:53.403 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:53.403 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:53.411 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:53.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:53.483 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:53.482+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:53.539 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:53.539 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:53.540 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:53.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:53.589 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:53.588+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:53.661 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:53.662 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:29:53.686 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:53.686+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:53.744 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:53.745 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:29:53.754 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.59881999999999, 2023-10-30T06:29:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:53.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:53.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:53.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:53.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:53.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:53.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:53.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 338.68564613699999, 2023-10-30T06:29:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:53.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:53.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:53.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:53.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:53.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:53.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 338.02546816799997, 2023-10-30T06:29:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:53.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:53.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.791 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 337.950044961, 2023-10-30T06:29:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:53.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:53.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:53.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:53.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:53.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:53.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:53.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:53.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:53.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:53.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.803 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:53.804 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:53.804 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 337.67892182999998, 2023-10-30T06:29:53.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:53.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:53.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:53.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:53.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 337.64864211999998, 2023-10-30T06:29:53.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:53.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:53.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:53.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:53.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:53.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:53.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:53.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:53.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.827 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.828 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 337.27473103599999, 2023-10-30T06:29:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:53.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:53.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:53.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:53.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:53.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:53.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:53.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:53.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:53.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:53.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:53.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.840 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.841 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:53.841 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:53.841 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 337.270232335, 2023-10-30T06:29:53.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:53.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.842 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:53.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:53.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:53.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:53.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:53.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:53.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:53.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:53.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:53.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:53.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:53.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:53.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:53.851 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.852 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.852 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.852 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:53.853 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:53.853 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.97862839599998, 2023-10-30T06:29:53.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:53.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.854 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:53.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:53.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:53.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:53.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:53.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:53.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:53.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:53.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:53.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:53.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:53.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:53.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:53.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:53.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:53.866 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.866 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.867 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.867 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:53.867 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.97269107900001, 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:53.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:53.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:53.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:53.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:53.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:53.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:53.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:53.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:53.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:53.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:53.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:53.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:53.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:53.878 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.878 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.878 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.879 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:53.879 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:53.879 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.97139206399999, 2023-10-30T06:29:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:53.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:53.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:53.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:53.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:53.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:53.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.886 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:53.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:53.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:53.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:53.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:53.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:53.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:53.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:53.890 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.890 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.890 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.891 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:53.891 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:53.891 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.86840325600002, 2023-10-30T06:29:53.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:53.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.892 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:53.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:53.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:53.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:53.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:53.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:53.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:53.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:53.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:53.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:53.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:53.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:53.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:53.902 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.902 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:53.903 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:53.904 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.59649998899999, 2023-10-30T06:29:53.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:53.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.905 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:53.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:53.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:53.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:53.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:53.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:53.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:53.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:53.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:53.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:53.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:53.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:53.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.915 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.916 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:53.916 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:53.916 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.58900179699998, 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:53.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:53.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:53.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:53.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:53.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:53.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:53.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:53.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:53.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:53.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:53.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:53.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:53.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:53.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:53.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:53.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:53.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:53.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.928 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.928 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.929 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:53.929 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:53.929 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.58834059200001, 2023-10-30T06:29:53.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:53.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.930 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:53.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:53.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:53.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:53.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:53.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:53.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:53.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:53.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:53.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:53.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.940 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.940 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.941 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:53.941 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:53.941 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.575754754, 2023-10-30T06:29:53.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:53.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.942 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:53.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:53.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:53.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:53.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:53.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:53.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:53.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:53.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:53.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:53.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:53.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:53.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:53.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:53.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:53.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.953 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.953 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.56966254100001, 2023-10-30T06:29:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:53.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.955 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:53.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:53.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:53.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:53.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:53.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:53.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:53.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:53.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:53.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:53.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:53.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:53.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:53.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:53.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:53.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:53.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:53.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:53.966 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.966 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.966 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.967 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:53.967 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:53.967 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 336.56866719999999, 2023-10-30T06:29:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:53.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.970 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:53.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:53.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:53.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:53.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:53.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:53.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:53.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:53.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:53.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:53.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:53.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:53.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:53.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.979 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.979 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.979 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:53.980 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:53.980 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 329.24599462899999, 2023-10-30T06:29:53.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:53.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.981 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:53.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:53.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:53.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:53.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:53.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:53.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:53.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:53.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:53.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:53.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:53.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:53.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:53.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:53.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:53.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.992 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:53.992 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:53.992 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 329.21255314400003, 2023-10-30T06:29:53.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:53.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:53.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:53.993 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:53.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:53.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:53.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:53.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:53.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:53.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:53.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:53.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:53.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:53.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:53.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:53.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:54.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:54.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:54.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:54.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:54.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:54.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:54.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:54.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:54.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:54.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:54.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:54.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:54.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:54.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:54.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:54.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:54.003 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:54.003 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:54.004 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:54.004 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:54.004 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:54.006 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:54.006 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.008 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:54.009 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:54.010 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:54.011 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.011 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:54.005+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:54.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:29:54.094 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:54.095 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.107 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:54.107+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:54.200 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:54.200 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.208 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:54.208 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:54.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:54.230 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:54.228+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:54.294 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:54.295 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:29:54.348 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:54.347+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:54.364 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:54.364 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:54.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:54.418 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:54.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:54.500 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:54.500 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:54.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:54.510 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:54.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:54.611 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:54.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:54.691 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:54.691 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:54.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:54.745 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:54.744+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:54.812 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:54.812+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:54.837 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:54.837 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:54.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:54.931 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:54.929+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:55.014 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:55.013+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:55.016 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.017 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:55.114 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:55.114+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:55.161 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.162 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:55.216 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:55.214+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:55.316 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:55.316+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:55.345 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.345 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:55.417 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:55.416+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:55.518 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:55.517+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:55.523 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.523 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:55.619 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:55.619+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:29:55.687 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.688 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:29:55.721 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:55.719+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:55.821 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:55.821+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:55.865 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:55.866 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:55.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:29:55.923 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:55.921+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:56.024 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:56.023+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:56.044 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.044 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.054 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:29:56.139 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:56.138+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:56.227 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.227 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.228 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:56.226+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:56.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:29:56.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:29:56.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:29:56.366 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:56.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:56.407 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.408 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:29:56.458 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:56.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:56.530 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:56.529+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:56.589 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.589 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:29:56.631 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:56.629+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:56.732 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:56.731+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:56.785 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.786 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:29:56.839 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:56.838+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:56.866 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:29:56.866 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:29:56.934 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:56.933+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:56.947 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:56.947 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:56.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:29:56.980 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:29:56.980 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.035 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:57.033+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:57.044 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:29:57.058 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:29:57.059 INFO:tasks.mon_recovery:removing mon c 2023-10-30T06:29:57.059 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T06:29:57.059 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:29:57.074 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:29:57.063+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 110783) UID: 0 2023-10-30T06:29:57.074 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:29:57.064+0000 9f64640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:29:57.087 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:29:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.137 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:57.137+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:57.141 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:29:57.141 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:29:57.152 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:29:57.215 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:29:57.215 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.237 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:57.236+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:57.320 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:29:57.320 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.329 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:57.329 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:57.329 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:57.330 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:57.337 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:57.337+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:57.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:29:57.352 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:29:57.352 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:29:57.353 INFO:tasks.thrashosds.thrasher:Adding osd 3 2023-10-30T06:29:57.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2023-10-30T06:29:57.439 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:57.438+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:57.440 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:29:57.440 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.513 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:57.513 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:57.514 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:57.514 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:57.514 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:57.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:29:57.539 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:57.539+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:57.542 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:29:57.542 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.640 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:57.639+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:57.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:29:57.670 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:29:57.670 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:29:57.672 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:57.673 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:57.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:57.673 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:57.673 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:57.681 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:29:57.766 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:57.764+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:57.842 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:29:57.840+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:29:57.850 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:57.850 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:57.851 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:57.851 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:57.859 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:29:57.860 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:49.075 110788== 2023-10-30T06:29:57.861 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:49.075 110788== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:29:57.872 INFO:tasks.ceph.mon.c.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:29:57.943 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:57.942+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:58.027 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:58.028 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:58.028 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:58.028 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:58.028 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:58.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:29:58.044 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:58.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:58.087 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:29:58.082+0000 ff6f640 -1 mon.a@0(leader).osd e103 definitely_dead 0 2023-10-30T06:29:58.145 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:58.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:58.209 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:58.209 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:58.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.210 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:29:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.05180441900001, 2023-10-30T06:29:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:29:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:29:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:29:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:29:58.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:58.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:29:58.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:29:58.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:29:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:29:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:29:58.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:29:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:29:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:29:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:29:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:29:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:29:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:29:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:29:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:29:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:29:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:29:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.223 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:29:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 343.13863055600001, 2023-10-30T06:29:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:29:58.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:29:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:29:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:29:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:29:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:29:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:29:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:29:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:29:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:29:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:29:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:29:58.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:29:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:29:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.47845258699999, 2023-10-30T06:29:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:29:58.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:29:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:29:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:29:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:29:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:29:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:29:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:29:58.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:29:58.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:29:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:29:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:29:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:29:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:29:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:29:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:29:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:29:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:58.253 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.40302938000002, 2023-10-30T06:29:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:29:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:58.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:29:58.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:29:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:29:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:29:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:29:58.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:29:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:29:58.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:29:58.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:29:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:29:58.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:29:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:29:58.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:29:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:29:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:29:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:29:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:29:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.266 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:58.267 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.131906249, 2023-10-30T06:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:29:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:29:58.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:29:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:29:58.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:29:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:29:58.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:29:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:29:58.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:29:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:29:58.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:29:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:29:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:29:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:29:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:29:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:29:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:29:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.279 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:29:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:58.281 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.10162653899999, 2023-10-30T06:29:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:29:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:29:58.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:29:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:29:58.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:29:58.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:29:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:29:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:29:58.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:29:58.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:29:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:29:58.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:29:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:29:58.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:29:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:29:58.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:29:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:29:58.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.296 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:29:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.72771545500001, 2023-10-30T06:29:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:29:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:58.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:29:58.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:29:58.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:29:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:29:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:29:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:29:58.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:29:58.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:29:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:29:58.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:29:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:29:58.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:29:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:29:58.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:29:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:29:58.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.308 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.72321675400002, 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.309 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:29:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:29:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:29:58.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:29:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:29:58.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:29:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:29:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:29:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:29:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:29:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:29:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.319 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:29:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:58.320 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.43161281499999, 2023-10-30T06:29:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:29:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.323 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:29:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:29:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.324 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:29:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:29:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:29:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:29:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:29:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:29:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:29:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:29:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:29:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:29:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:29:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.42567549799998, 2023-10-30T06:29:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:29:58.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:29:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.335 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.336 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:29:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:29:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:29:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:29:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:29:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:29:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:29:58.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:29:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:29:58.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:29:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:29:58.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:29:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:29:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.424376483, 2023-10-30T06:29:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:29:58.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:29:58.347 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:29:58.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:29:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:29:58.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:29:58.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:29:58.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:29:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:29:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:29:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:29:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.354 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:29:58.355 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.32138767499998, 2023-10-30T06:29:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:29:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.356 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:29:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:29:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:29:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:29:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:29:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:29:58.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:29:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:29:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:29:58.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:29:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:29:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:29:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:29:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:29:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:29:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:29:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:29:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.370 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:29:58.371 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.04948440800001, 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:29:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:29:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:29:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:29:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:29:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:29:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:29:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:29:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:29:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:29:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:29:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:29:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:29:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:29:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:29:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:29:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:29:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:29:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:58.383 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.041986216, 2023-10-30T06:29:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:29:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:29:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:29:58.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:29:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:29:58.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:29:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:29:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:29:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:29:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:29:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:29:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:29:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:29:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:29:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.04132501100003, 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:58.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:29:58.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:29:58.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:29:58.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:29:58.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:29:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:29:58.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:29:58.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:29:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:29:58.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:29:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:29:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.406 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:58.407 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.02873917300002, 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:29:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:29:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:29:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:29:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:29:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:29:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:29:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:29:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:29:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:29:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:29:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:29:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:29:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:29:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.02264695999997, 2023-10-30T06:29:58.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:29:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:29:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:29:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:29:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:29:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:29:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:29:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:29:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:29:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:29:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:29:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:29:58.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:29:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:29:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:29:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:29:58.431 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.432 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.433 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:29:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 341.02165161900001, 2023-10-30T06:29:58.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:29:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.435 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:29:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:29:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:29:58.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:29:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:29:58.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:29:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:29:58.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:29:58.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:29:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:29:58.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:29:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:29:58.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:29:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:29:58.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:29:58.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.448 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:29:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 333.69897904800001, 2023-10-30T06:29:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:29:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:58.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:29:58.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:29:58.452 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:29:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.453 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:29:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:29:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:29:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:29:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:29:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:29:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:29:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:29:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:29:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:29:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:29:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 333.66553756299999, 2023-10-30T06:29:58.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:29:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:29:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:29:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:58.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:29:58.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:29:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:58.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:29:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:29:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:58.467 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:29:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:29:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:58.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:29:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:29:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:58.469 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:29:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:29:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:29:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:29:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:29:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.471 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:29:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:29:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:29:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:58.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:29:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:29:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:58.474 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:58.474 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:58.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:29:58.477 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:29:58.475+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:29:58.578 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:58.576+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:58.606 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:58.606 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:58.606 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:58.606 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:58.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:29:58.678 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:58.678+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:58.779 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:58.779+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:58.782 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:58.783 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:58.783 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:58.783 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:58.784 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:58.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:29:58.881 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:58.880+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:58.919 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:58.919 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:58.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:58.920 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:29:58.920 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:58.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:29:58.982 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:29:58.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:29:59.055 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:59.055 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:59.055 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:29:59.056 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:59.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:29:59.083 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:59.083+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:59.184 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:59.183+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:59.230 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:59.230 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:29:59.230 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:29:59.230 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:29:59.231 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:59.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:29:59.284 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:29:59.283+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:29:59.368 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:29:59.369 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:29:59.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:29:59.369 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:29:59.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.369 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:29:59.370 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:59.370 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 262.39439055499997, 2023-10-30T06:29:59.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:29:59.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:59.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:59.371 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:59.371 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:59.371 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:59.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:29:59.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.372 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:59.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:59.372 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:29:59.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:59.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:59.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:29:59.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:29:59.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:59.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:29:59.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:29:59.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:59.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:29:59.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:29:59.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:59.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:29:59.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:29:59.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:59.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:29:59.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:29:59.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:59.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.381 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:29:59.381 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:59.381 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 82.373898230999998, 2023-10-30T06:29:59.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:29:59.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:29:59.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:29:59.382 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:29:59.382 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:29:59.382 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:29:59.383 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:29:59.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:29:59.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:29:59.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:29:59.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:29:59.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:29:59.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:29:59.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:29:59.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:29:59.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:29:59.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:29:59.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:29:59.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:59.390 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:59.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:59.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:29:59.391 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:29:59.391 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:29:59.392 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:59.391+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:59.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:29:59.493 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:29:59.492+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:29:59.529 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:59.530 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:29:59.530 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:59.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:29:59.594 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:59.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:59.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:29:59.708 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:29:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:29:59.708 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:29:59.709 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:59.709 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:29:59.707+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:29:59.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:29:59.802 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:59.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:29:59.887 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:29:59.887 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:29:59.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:29:59.888 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:29:59.888 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:29:59.896 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:29:59.898 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:29:59.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:29:59.999 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:29:59.997+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:00.064 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:00.064 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:00.065 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:00.065 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:00.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:00.100 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:00.099+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:00.202 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:00.200+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:00.247 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:00.247 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:00.248 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:00.248 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:00.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:00.341 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:00.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:00.403 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:00.402+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:00.478 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:00.478 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:00.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:00.479 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 347.14143401899997, 2023-10-30T06:30:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.480 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:00.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:00.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:00.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:00.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:00.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:00.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.497 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.82027882800003, 2023-10-30T06:30:00.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.499 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.500 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.81603427099998, 2023-10-30T06:30:00.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.515 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:00.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:00.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:00.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:00.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:00.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:00.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.529 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.73057584600002, 2023-10-30T06:30:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.530 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:00.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:00.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:00.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:00.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:00.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:00.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:00.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:00.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:00.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:00.545 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.46133263000002, 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.546 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:00.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:00.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:00.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:00.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:00.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:00.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:00.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:00.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.560 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.46050121899998, 2023-10-30T06:30:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.561 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.562 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.45983409199999, 2023-10-30T06:30:00.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.577 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:00.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:00.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:00.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:00.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:00.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:00.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:00.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:00.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:00.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:00.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:00.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.591 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.43756002100002, 2023-10-30T06:30:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.592 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.593 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:00.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:00.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.43691216500002, 2023-10-30T06:30:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:00.607 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:00.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:00.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:00.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:00.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:00.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:00.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:00.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.621 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:00.622 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.43622609200003, 2023-10-30T06:30:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.623 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 343.36534412399999, 2023-10-30T06:30:00.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.639 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:00.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:00.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:00.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:00.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 343.364680158, 2023-10-30T06:30:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:00.655 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:00.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:00.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:00.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:00.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.669 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:00.670 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 343.33622634900001, 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.671 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.675 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.677 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:00.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:00.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:00.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:00.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:00.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:00.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.99445985599999, 2023-10-30T06:30:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:00.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:00.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:00.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:00.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:00.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:00.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:00.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:00.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:00.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:00.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.99371118599998, 2023-10-30T06:30:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.703 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.713 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.99304280299998, 2023-10-30T06:30:00.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.718 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:00.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:00.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:00.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:00.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:00.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.98319212899997, 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.733 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:00.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:00.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:00.736 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:00.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:00.738 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:00.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:00.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:00.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:00.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:00.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:00.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:00.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:00.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:00.747 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.748 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.982518314, 2023-10-30T06:30:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.749 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.750 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:00.764 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.98186993100001, 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.765 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.767 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.768 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.769 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.770 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 335.97775346499998, 2023-10-30T06:30:00.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:00.781 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:00.785 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.786 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.787 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:00.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:00.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:00.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:00.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:00.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:00.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:00.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:00.795 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:00.796 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:00.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:00.798 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:00.796+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:00.898 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:00.896+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:00.971 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:00.971 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:00.971 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:00.971 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:00.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:01.026 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:01.024+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:01.100 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:01.098+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:01.112 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:01.113 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:01.113 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:01.113 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:01.113 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:01.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:01.201 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:01.199+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:01.289 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:01.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:01.290 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:01.290 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:01.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:01.302 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:01.300+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:01.402 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:01.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:01.426 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:01.426 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:01.426 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:01.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:01.503 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:01.501+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:01.562 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:01.563 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:01.563 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:01.563 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:01.563 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:01.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:01.614 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:01.611+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:01.705 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:01.703+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:01.741 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:01.741 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:01.742 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 265.447619729, 2023-10-30T06:30:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:01.743 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:01.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.745 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:01.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:01.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:01.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:01.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:01.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:01.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:01.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:01.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:01.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:01.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:01.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:01.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:01.752 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:01.752 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:01.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:01.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:01.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 85.415445509999998, 2023-10-30T06:30:01.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:01.754 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:01.754 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:01.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:01.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:01.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:01.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:01.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:01.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:01.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:01.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:01.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:01.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:01.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:01.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:01.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:01.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:01.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:01.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:01.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:01.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:01.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:01.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:01.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:01.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:01.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:01.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:01.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:01.763 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:01.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:01.763 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:01.764 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:01.764 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:01.765 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:01.807 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:01.805+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:01.906 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:01.905+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:01.909 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:01.910 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:01.910 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:01.910 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:01.920 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:02.009 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:02.007+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:02.047 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:02.048 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:02.048 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:02.048 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:02.048 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:02.057 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:02.108 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:02.106+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:02.182 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:02.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:02.183 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:02.183 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:02.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:02.208 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:02.207+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:02.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-30T06:30:02.309 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:02.307+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:02.357 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:02.357 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:02.357 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:02.358 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:02.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:02.409 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:02.408+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:02.510 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:02.509+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:02.531 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:02.531 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:02.532 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:02.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:02.611 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:02.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:02.711 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:02.712 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:02.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:02.712 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:02.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.712 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.55479823899998, 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:02.713 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:02.714 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:02.714 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:02.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:02.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.715 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:02.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:02.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:02.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:02.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:02.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:02.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:02.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:02.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:02.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:02.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:02.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.724 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.725 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:02.725 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:02.725 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 347.64162437599998, 2023-10-30T06:30:02.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:02.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.726 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:02.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:02.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:02.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:02.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:02.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:02.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:02.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:02.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:02.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:02.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:02.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:02.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:02.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:02.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.736 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.737 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:02.737 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:02.738 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.98144640700002, 2023-10-30T06:30:02.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:02.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.738 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:02.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:02.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:02.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.742 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.742 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:02.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:02.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:02.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:02.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:02.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:02.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:02.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:02.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:02.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:02.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:02.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:02.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:02.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:02.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:02.749 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:02.750 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.90602319999999, 2023-10-30T06:30:02.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:02.750 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.750 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:02.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:02.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:02.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:02.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:02.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:02.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:02.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:02.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:02.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:02.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:02.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:02.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.760 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.761 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.761 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:02.761 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:02.762 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.63490006900003, 2023-10-30T06:30:02.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:02.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:02.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:02.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:02.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:02.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:02.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:02.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:02.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:02.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:02.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:02.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:02.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:02.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:02.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:02.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:02.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.774 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:02.774 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:02.774 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.60462035900002, 2023-10-30T06:30:02.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:02.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.775 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:02.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:02.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:02.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:02.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:02.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:02.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:02.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:02.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:02.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:02.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:02.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:02.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:02.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:02.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.785 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.785 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.786 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:02.786 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:02.786 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.23070927499998, 2023-10-30T06:30:02.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:02.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.787 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:02.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:02.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:02.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:02.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:02.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:02.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:02.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:02.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:02.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:02.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:02.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:02.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:02.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:02.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.797 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.797 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.798 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:02.798 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:02.798 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 346.22621057399999, 2023-10-30T06:30:02.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:02.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.799 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:02.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:02.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:02.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:02.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:02.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:02.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:02.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:02.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:02.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:02.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:02.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:02.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:02.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:02.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:02.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:02.809 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.809 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.810 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.810 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:02.810 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:02.811 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.93460663500002, 2023-10-30T06:30:02.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:02.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.811 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:02.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:02.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:02.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:02.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:02.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:02.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:02.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:02.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:02.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:02.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:02.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:02.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:02.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.822 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:02.822 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:02.822 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.928669318, 2023-10-30T06:30:02.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:02.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.823 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:02.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:02.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:02.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:02.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:02.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:02.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:02.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:02.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:02.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:02.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:02.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:02.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:02.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:02.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:02.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:02.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.833 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.834 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:02.834 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:02.834 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.92737030299998, 2023-10-30T06:30:02.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:02.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:02.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:02.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:02.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:02.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:02.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:02.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:02.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:02.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:02.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:02.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:02.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:02.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:02.845 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.845 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.845 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.82438149500001, 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.847 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:02.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:02.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:02.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:02.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:02.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:02.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:02.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:02.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.856 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:02.857 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.55247822799998, 2023-10-30T06:30:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.858 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.54498003600003, 2023-10-30T06:30:02.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:02.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:02.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.880 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.544318831, 2023-10-30T06:30:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:02.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:02.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:02.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:02.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:02.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.53173299299999, 2023-10-30T06:30:02.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:02.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:02.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:02.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:02.902 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:02.903 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.52564078, 2023-10-30T06:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.904 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:02.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 345.52464543899998, 2023-10-30T06:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:02.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:02.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:02.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:02.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:02.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:02.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.925 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 338.20197286799998, 2023-10-30T06:30:02.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:02.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:02.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.937 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 338.16853138300002, 2023-10-30T06:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:02.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:02.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:02.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:02.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:02.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:02.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:02.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:02.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:02.949 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:02.949 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:02.950 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:02.951 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:02.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:03.051 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:03.050+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:03.116 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:03.117 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:03.117 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:03.161 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:30:03.162 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-30T06:30:03.162 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T06:30:03.163 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-10-30T06:30:03.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:03.170 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:03.169+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:03.252 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.252 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:03.252 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:03.253 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:03.253 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.253 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:03.251+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:03.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:03.353 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:03.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:03.426 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.427 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:03.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:03.427 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:03.427 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:03.456 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:03.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:03.555 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:03.554+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:03.562 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.562 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:03.563 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:03.563 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.571 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:03.656 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:03.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:03.696 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.697 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:03.697 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:03.697 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:03.697 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:03.757 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:03.755+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:03.790 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:03.837 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:03.837 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:03.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:03.838 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:03.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.838 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:03.839 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:03.839 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 266.86336959900001, 2023-10-30T06:30:03.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:03.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:03.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:03.840 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:03.840 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:03.840 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:03.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:03.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.841 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:03.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:03.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:03.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:03.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:03.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:03.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:03.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:03.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:03.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:03.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:03.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:03.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:03.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:03.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:03.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:03.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:03.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:03.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:03.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:03.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:03.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:03.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:03.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:03.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:03.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:03.849 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:03.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:03.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.850 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:03.850 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:03.850 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 86.842877275000006, 2023-10-30T06:30:03.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:03.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:03.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:03.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:03.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:03.851 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:03.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:03.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:03.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:03.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:03.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:03.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:03.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:03.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:03.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:03.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:03.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:03.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:03.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:03.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:03.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:03.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:03.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:03.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:03.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:03.860 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:03.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:03.863 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:03.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:03.963 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:03.961+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:03.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:03.990 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:03.990 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:04.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:04.064 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:04.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:04.165 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:04.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:04.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:04.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:04.266 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:04.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:04.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:04.348 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:04.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:04.348 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:04.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:04.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:04.366 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:04.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:04.467 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:04.466+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:04.514 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:04.515 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:04.515 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:04.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:04.574 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:04.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:04.647 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:04.647 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:04.647 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:04.647 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:04.648 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:04.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:04.699 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:04.696+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:04.771 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:04.769+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:04.872 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:04.871+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:04.879 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:04.879 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 351.54244152699999, 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:04.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.881 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:04.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:04.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:04.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:04.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:04.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:04.896 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.897 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 349.22128633599999, 2023-10-30T06:30:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.898 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.899 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:04.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.912 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 349.217041779, 2023-10-30T06:30:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.913 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:04.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:04.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 349.13158335399999, 2023-10-30T06:30:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:04.928 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.929 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:04.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:04.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:04.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:04.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:04.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:04.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.942 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.943 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.86234013799998, 2023-10-30T06:30:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.944 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.945 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:04.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:04.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.861508727, 2023-10-30T06:30:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:04.959 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.960 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:04.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:04.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:04.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.974 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.975 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.86084160000001, 2023-10-30T06:30:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.976 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.977 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:04.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:04.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:04.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:04.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:04.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.83856752899999, 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.83791967299999, 2023-10-30T06:30:05.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.008 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:05.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:05.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:05.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:05.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:05.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:05.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:05.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:05.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:05.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.021 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:05.022 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 348.83723359999999, 2023-10-30T06:30:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.023 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.024 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:05.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:05.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.037 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:05.038 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 347.76635163200001, 2023-10-30T06:30:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.039 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 347.76568766600002, 2023-10-30T06:30:05.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.056 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.068 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.069 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.070 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:05.071 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 347.73723385699998, 2023-10-30T06:30:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.072 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.073 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:05.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:05.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:05.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:05.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:05.083 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.085 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.086 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.087 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.39546736400001, 2023-10-30T06:30:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.088 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.100 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:05.102 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.103 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.39471869400001, 2023-10-30T06:30:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.104 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.105 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:05.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:05.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:05.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:05.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:05.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.118 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:05.119 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.394050311, 2023-10-30T06:30:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.120 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.130 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.131 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.132 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.38419963699999, 2023-10-30T06:30:05.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.136 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:05.145 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.146 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.147 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:05.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:05.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.38352582200002, 2023-10-30T06:30:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:05.151 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.152 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.153 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.160 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.38287743900003, 2023-10-30T06:30:05.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.169 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:05.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:05.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:05.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.178 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.179 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.180 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:05.181 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 340.378760973, 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:05.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:05.186 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:05.187 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:05.188 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:05.189 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:05.190 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:05.191 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:05.192 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:05.193 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:05.194 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:05.195 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:05.196 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.197 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:05.198 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:05.198 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:05.200 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:05.198+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:05.301 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:05.300+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:05.329 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:05.329 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:05.329 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:05.402 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:05.401+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:05.465 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:05.465 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:05.466 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:05.466 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:05.503 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:05.502+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:05.603 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:05.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:05.604 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:05.604 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.605 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:05.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:05.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:05.705 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:05.703+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:05.782 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:05.782 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:05.783 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:05.783 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:05.836 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:05.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:05.907 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:05.905+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:05.919 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:05.919 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:05.920 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:05.920 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:05.928 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:06.008 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:06.006+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:06.103 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:06.103 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:06.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:06.103 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:06.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.104 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:06.104 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:06.104 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 269.80919689500001, 2023-10-30T06:30:06.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:06.105 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:06.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:06.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 89.777022676000001, 2023-10-30T06:30:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:06.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:06.117 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:06.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:06.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:06.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:06.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:06.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:06.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:06.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:06.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:06.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:06.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:06.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:06.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:06.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:06.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:06.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:06.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:06.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:06.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:06.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:06.125 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:06.125 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:06.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:06.126 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:06.126 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:06.126 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:06.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:06.128 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:06.126+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:06.229 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:06.227+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:06.265 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:06.266 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:06.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:06.330 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:06.328+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:06.401 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:06.402 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:06.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:06.454 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:06.452+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:06.532 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:06.531+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:06.535 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:06.536 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:06.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:06.634 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:06.632+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:06.711 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:06.711 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:06.712 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:06.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:06.763 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:06.762+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:06.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:06.835 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:06.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:06.845 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:06.845 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:06.845 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:06.845 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:06.846 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:06.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:06.938 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:06.937+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:07.038 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:07.036+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:07.062 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:07.063 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.90658189800001, 2023-10-30T06:30:07.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:07.065 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.066 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:07.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:07.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:07.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:07.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.078 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:07.079 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.080 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.081 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 351.99340803500002, 2023-10-30T06:30:07.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:07.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:07.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:07.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:07.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:07.089 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:07.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:07.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:07.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:07.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 351.333230066, 2023-10-30T06:30:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:07.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:07.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:07.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:07.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:07.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:07.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:07.102 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:07.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:07.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:07.105 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.106 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 351.25780685900003, 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.107 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:07.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:07.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:07.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.113 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:07.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:07.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:07.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:07.117 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.118 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.986683728, 2023-10-30T06:30:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:07.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:07.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:07.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:07.123 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:07.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:07.125 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:07.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:07.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:07.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:07.129 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.130 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.956404018, 2023-10-30T06:30:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:07.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:07.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:07.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.135 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:07.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:07.137 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:07.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:07.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:07.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:07.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.58249293400002, 2023-10-30T06:30:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:07.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.147 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.148 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:07.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:07.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.57799423300003, 2023-10-30T06:30:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:07.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.160 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.161 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.167 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:07.167 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:07.167 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.286390294, 2023-10-30T06:30:07.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:07.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.168 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:07.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:07.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:07.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:07.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.171 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.171 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.172 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:07.172 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:07.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.172 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.173 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:07.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:07.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.173 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:07.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:07.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:07.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:07.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:07.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:07.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:07.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:07.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.28045297699998, 2023-10-30T06:30:07.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:07.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.180 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:07.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:07.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:07.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:07.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.183 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.184 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:07.184 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:07.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.185 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:07.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:07.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:07.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:07.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:07.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:07.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:07.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:07.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:07.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:07.190 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.190 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.191 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.191 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:07.191 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:07.192 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.27915396200001, 2023-10-30T06:30:07.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:07.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.192 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:07.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.196 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.196 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:07.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:07.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.197 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.197 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:07.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:07.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:07.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:07.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:07.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:07.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:07.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:07.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:07.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:07.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.202 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.202 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.203 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:07.203 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:07.203 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 350.17616515399999, 2023-10-30T06:30:07.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:07.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.204 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:07.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:07.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:07.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:07.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:07.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:07.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:07.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:07.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:07.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:07.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:07.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:07.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:07.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:07.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.214 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.214 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.215 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:07.215 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:07.215 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.90426188700002, 2023-10-30T06:30:07.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:07.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.216 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:07.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:07.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:07.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:07.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:07.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:07.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:07.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.222 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:07.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:07.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:07.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:07.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:07.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.896763695, 2023-10-30T06:30:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.228 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:07.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:07.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:07.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:07.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:07.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:07.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:07.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:07.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:07.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:07.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:07.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:07.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.238 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.238 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.239 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:07.239 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:07.239 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.89610248999998, 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:07.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:07.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:07.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:07.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:07.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:07.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.88351665200003, 2023-10-30T06:30:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.262 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.87742443899998, 2023-10-30T06:30:07.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.264 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.265 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.271 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.274 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:07.275 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 349.87642909800002, 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.277 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.286 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.55375652700002, 2023-10-30T06:30:07.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.298 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:07.299 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 342.52031504199999, 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:07.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:07.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:07.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:07.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:07.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.310 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:07.311 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:07.311 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:07.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:07.313 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:07.312+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:07.414 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:07.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:07.480 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:07.480 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:07.481 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:07.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:07.532 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:07.531+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:07.615 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:07.615 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:07.616 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:07.616 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:07.616 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:07.619 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:07.617+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:07.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:07.719 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:07.718+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:07.792 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:07.792 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:07.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:07.793 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:07.793 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:07.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:07.821 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:07.819+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:07.922 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:07.920+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:07.930 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:07.930 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:07.930 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:07.930 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:07.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:08.024 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:08.023+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:08.106 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:08.106 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:08.107 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:08.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:08.157 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:08.156+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:08.226 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:08.225+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:08.283 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:08.284 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:08.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:08.284 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:08.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.285 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:08.285 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:08.285 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 271.30979555699997, 2023-10-30T06:30:08.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:08.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:08.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:08.286 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:08.286 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:08.286 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:08.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:08.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.287 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:08.287 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:08.288 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:08.288 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:08.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.289 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:08.289 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:08.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:08.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:08.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:08.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:08.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:08.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:08.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:08.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:08.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:08.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:08.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:08.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:08.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:08.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:08.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:08.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:08.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:08.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:08.296 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:08.296 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:08.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.297 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:08.297 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:08.297 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 91.289303232999998, 2023-10-30T06:30:08.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:08.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:08.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:08.298 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:08.298 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:08.298 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:08.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:08.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.299 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:08.299 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.299 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:08.300 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:08.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:08.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.300 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:08.301 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:08.301 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:08.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:08.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:08.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:08.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:08.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:08.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:08.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:08.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:08.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:08.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:08.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:08.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:08.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:08.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:08.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:08.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:08.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:08.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:08.307 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:08.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:08.308 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:08.308 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:08.308 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:08.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:08.354 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:08.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:08.428 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:08.427+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:08.448 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:08.449 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:08.449 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:08.449 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:08.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:08.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-30T06:30:08.529 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:08.528+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:08.624 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:08.624 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:08.625 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:08.625 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:08.625 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:08.630 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:08.628+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:08.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:08.731 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:08.729+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:08.804 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:08.805 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:08.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:08.805 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:08.805 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:08.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:08.831 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:08.830+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:08.933 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:08.931+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:08.981 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:08.981 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:08.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:09.034 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:09.032+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:09.119 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:09.120 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:09.121 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:09.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:09.135 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:09.134+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:09.236 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:09.235+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:09.339 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:09.337+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:09.353 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:09.353 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:09.354 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 356.01392610699997, 2023-10-30T06:30:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.355 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.356 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:09.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:09.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:09.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:09.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:09.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.373 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:09.374 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.69277091599997, 2023-10-30T06:30:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.375 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:09.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:09.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:09.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:09.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:09.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:09.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:09.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.68852635899998, 2023-10-30T06:30:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:09.407 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.60306793400002, 2023-10-30T06:30:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:09.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:09.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:09.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.422 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.33382471800002, 2023-10-30T06:30:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.423 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:09.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:09.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:09.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:09.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:09.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:09.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:09.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:09.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:09.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:09.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:09.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:09.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.437 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.438 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.33299330699998, 2023-10-30T06:30:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.439 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.440 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:09.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.33232618, 2023-10-30T06:30:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:09.454 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:09.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:09.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:09.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:09.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:09.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:09.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:09.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:09.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:09.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:09.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:09.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:09.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:09.468 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.469 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.31005210900003, 2023-10-30T06:30:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.470 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:09.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:09.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:09.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:09.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:09.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:09.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:09.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:09.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:09.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:09.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:09.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:09.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:09.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:09.484 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.485 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.30940425300003, 2023-10-30T06:30:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.486 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:09.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:09.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:09.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:09.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:09.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:09.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:09.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:09.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:09.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:09.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.30871818000003, 2023-10-30T06:30:09.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.502 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:09.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:09.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:09.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:09.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:09.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:09.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:09.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:09.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:09.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:09.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:09.515 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 352.23783621199999, 2023-10-30T06:30:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.517 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:09.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:09.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:09.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:09.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:09.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:09.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:09.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:09.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:09.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:09.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:09.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:09.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:09.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 352.237172246, 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.533 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.534 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:09.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:09.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:09.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:09.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:09.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:09.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:09.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:09.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:09.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:09.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 352.20871843700002, 2023-10-30T06:30:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:09.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.550 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:09.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:09.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:09.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:09.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:09.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:09.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:09.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:09.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.564 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.564 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:09.564 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.86695194399999, 2023-10-30T06:30:09.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.565 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:09.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:09.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.86620327399999, 2023-10-30T06:30:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:09.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.581 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:09.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:09.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:09.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:09.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:09.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:09.594 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.595 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.86553489099998, 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.596 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.597 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.602 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.603 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.604 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:09.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.85568421699998, 2023-10-30T06:30:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:09.613 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.614 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:09.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:09.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:09.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:09.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:09.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:09.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:09.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:09.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:09.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:09.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:09.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.627 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.628 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.855010402, 2023-10-30T06:30:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.629 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.630 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.636 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:09.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:09.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:09.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:09.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:09.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:09.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:09.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:09.642 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.643 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:09.644 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.85436201900001, 2023-10-30T06:30:09.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:09.644 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.644 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.644 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.645 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.645 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:09.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.645 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:09.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:09.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.650 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:09.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:09.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.650 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.650 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.651 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:09.652 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:09.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:09.653 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:09.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:09.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:09.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.657 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.658 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:09.658 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:09.658 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 344.85024555299998, 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:09.659 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:09.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:09.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:09.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:09.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:09.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:09.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:09.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:09.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:09.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:09.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:09.665 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:09.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.666 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.666 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:09.666 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:09.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:09.666 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.667 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:09.667 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:09.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:09.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.668 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.668 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:09.668 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:09.668 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:09.668 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:09.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:09.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:09.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.670 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:09.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.672 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:09.673 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:09.673 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:09.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:09.675 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:09.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:09.776 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:09.774+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:09.844 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:09.845 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:09.845 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:09.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:09.877 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:09.875+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:09.979 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:09.978+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:09.982 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:09.982 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:09.982 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:09.982 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:09.983 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:09.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:10.079 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:10.078+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:10.124 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:10.124 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:10.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:10.125 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:10.125 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:10.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:10.180 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:10.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:10.261 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:10.261 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:10.261 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:10.261 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:10.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:10.281 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:10.280+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:10.382 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:10.380+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:10.437 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:10.438 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:10.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:10.490 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:10.488+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:10.577 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 274.28346899500002, 2023-10-30T06:30:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 94.251294775999995, 2023-10-30T06:30:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:10.599 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:10.601 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:10.602 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:10.599+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:10.702 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:10.700+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:10.740 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:10.741 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:10.741 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:10.751 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:10.836 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:10.834+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:10.903 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:10.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:10.923 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:10.924 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:10.924 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:10.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:11.016 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:11.015+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:11.099 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:11.099 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:11.100 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:11.100 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:11.105 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:11.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:11.108 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:11.205 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:11.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:11.273 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:11.274 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:11.275 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:11.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:11.327 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:11.326+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:11.407 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:11.405+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:11.409 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:11.410 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:11.410 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:11.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:11.507 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:11.506+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:11.609 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:11.607+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:11.625 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:11.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.46998875000003, 2023-10-30T06:30:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:11.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:11.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:11.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:11.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:11.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:11.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:11.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.638 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:11.639 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 356.55681488699997, 2023-10-30T06:30:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.643 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:11.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.89663691800001, 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:11.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:11.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:11.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:11.655 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:11.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:11.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:11.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:11.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:11.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:11.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.662 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.82121371099998, 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.663 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:11.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:11.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.668 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:11.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.673 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.674 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.55009058000002, 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.675 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:11.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:11.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:11.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:11.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:11.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.685 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:11.686 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.51981087000001, 2023-10-30T06:30:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.687 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.697 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:11.698 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.14589978599997, 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 355.14140108499998, 2023-10-30T06:30:11.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:11.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.84979714600001, 2023-10-30T06:30:11.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:11.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:11.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:11.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:11.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:11.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.733 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.843859829, 2023-10-30T06:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:11.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:11.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:11.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:11.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:11.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:11.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.744 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:11.745 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.84256081400002, 2023-10-30T06:30:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.746 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:11.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:11.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:11.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:11.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:11.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:11.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:11.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:11.756 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:11.757 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.739572006, 2023-10-30T06:30:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:11.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:11.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:11.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:11.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:11.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:11.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:11.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:11.768 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.769 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.46766873899998, 2023-10-30T06:30:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:11.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:11.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:11.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:11.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:11.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:11.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.780 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:11.781 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.46017054700002, 2023-10-30T06:30:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.782 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.45950934199999, 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:11.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:11.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:11.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:11.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:11.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:11.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:11.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:11.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.804 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.44692350399998, 2023-10-30T06:30:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:11.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:11.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:11.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:11.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:11.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:11.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:11.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:11.815 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.816 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.44083129099999, 2023-10-30T06:30:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:11.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 354.43983594999997, 2023-10-30T06:30:11.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:11.841 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 347.11716337899998, 2023-10-30T06:30:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:11.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:11.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:11.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 347.08372189400001, 2023-10-30T06:30:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:11.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:11.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:11.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:11.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:11.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:11.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:11.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:11.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:11.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:11.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:11.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:11.864 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:11.865 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:11.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:11.867 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:11.865+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:11.968 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:11.966+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:11.994 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:11.995 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:11.995 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:11.995 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:12.068 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:12.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:12.134 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:12.134 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:12.134 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:12.134 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:12.135 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:12.169 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:12.168+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:12.256 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:30:12.143+0000 11f74640 -1 mon.a@0(leader).osd e103 definitely_dead 0 2023-10-30T06:30:12.271 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:12.271 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:12.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:12.271 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:12.272 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.272 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:12.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:12.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:12.280 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:12.281 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.366 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:12.367 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.371 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:12.369+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:12.452 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:12.452 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.453 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:12.453 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:12.471 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:12.470+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:12.542 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:12.543 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.572 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:12.571+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:12.662 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:12.662 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.663 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:12.663 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:12.664 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:12.664 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:12.664 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:12.679 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:12.674+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:12.753 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:12.753 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.774 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:12.772+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:12.813 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.814 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:12.815 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:12.815 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 275.83939348500002, 2023-10-30T06:30:12.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:12.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:12.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:12.816 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:12.816 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:12.816 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:12.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:12.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:12.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:12.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:12.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:12.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:12.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:12.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:12.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:12.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:12.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:12.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:12.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:12.824 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:12.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:12.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.825 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:12.825 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:12.825 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 95.818901160999999, 2023-10-30T06:30:12.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:12.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:12.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:12.826 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:12.826 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:12.827 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:12.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:12.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.827 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:12.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:12.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:12.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:12.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:12.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:12.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:12.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:12.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:12.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:12.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:12.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:12.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:12.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:12.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:12.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:12.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:12.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:12.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:12.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:12.835 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:12.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:12.835 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:12.836 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:12.836 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:12.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:12.848 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:12.848 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.875 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:12.873+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:12.932 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:12.933 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:12.973 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:12.973 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:12.975 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:12.973+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:12.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:13.076 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:13.075+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:13.159 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.160 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:13.177 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:13.176+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:13.278 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:13.277+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:13.300 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.300 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:13.358 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.3. 2023-10-30T06:30:13.371 INFO:tasks.thrashosds.thrasher:Added osd 3 2023-10-30T06:30:13.379 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:13.377+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:13.430 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:30:13.446 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.446 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:13.486 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:13.484+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:13.538 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:13.553 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-10-30T06:30:13.553 INFO:tasks.mon_recovery:causing some monitor log activity 2023-10-30T06:30:13.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2023-10-30T06:30:13.581 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:13.579+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:13.634 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.634 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:13.681 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:13.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:13.783 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:13.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:13.812 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.812 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:13.883 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:13.882+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:13.949 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:13.949 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:13.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:14.006 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:14.004+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:14.081 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:45.962 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.085 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:14.084+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:14.094 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:14.094 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:14.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:14.131 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:46.011 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.162 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:01:46.043 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.179 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:40.514 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.183 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:40.518 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.186 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:14.184+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:14.205 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:00:40.540 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:14.267 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:14.267 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:14.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:14.287 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:14.285+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:14.388 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:14.386+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:14.466 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:14.467 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:14.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:14.489 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:14.487+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:14.607 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:14.605+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:14.690 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:14.689+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:14.760 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:14.761 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:14.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:14.811 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:14.810+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:14.893 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:14.891+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:14.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:14.959 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:14.959 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:14.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:15.013 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:15.011+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:15.094 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:15.092+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:15.099 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:15.100 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:15.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:15.183 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:15.183 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.195 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:15.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:15.268 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:15.268 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.277 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:15.277 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:15.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:15.295 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:15.294+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:15.354 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:15.355 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.396 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:15.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:15.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:30:15.457 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:15.458 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.459 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:15.459 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:15.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:15.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2023-10-30T06:30:15.515 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:15.514+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:15.547 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:15.548 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.598 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:15.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:15.651 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:15.651 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.652 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:15.652 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:15.652 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:15.652 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:15.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:15.699 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:15.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:15.737 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:15.737 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.800 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:15.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:15.830 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:15.831 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:15.831 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:15.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:15.850 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:15.851 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:15.904 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:15.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:16.002 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:16.001+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:16.014 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 363.85344129700002, 2023-10-30T06:30:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:16.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:16.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:16.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.023 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.023 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:16.023 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:16.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:16.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:16.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:16.025 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:16.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:16.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:16.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:16.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:16.027 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.028 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.028 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:16.029 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:16.029 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 360.94026743400002, 2023-10-30T06:30:16.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:16.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.030 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:16.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:16.032 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.032 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.032 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:16.033 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:16.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.033 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:16.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:16.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:16.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:16.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.036 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.036 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:16.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:16.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.037 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.037 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.037 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:16.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:16.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.038 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:16.038 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:16.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:16.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:16.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.040 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 360.280089465, 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:16.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.042 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:16.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:16.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:16.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:16.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:16.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:16.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:16.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:16.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.048 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.048 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:16.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:16.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.050 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:16.050 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:16.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:16.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:16.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:16.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:16.052 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.052 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.053 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.053 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:16.053 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:16.054 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 360.20466625799997, 2023-10-30T06:30:16.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:16.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.054 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:16.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:16.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:16.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:16.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.061 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:16.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:16.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:16.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:16.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:16.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:16.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.064 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.065 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:16.065 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:16.065 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.93354312700001, 2023-10-30T06:30:16.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:16.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.066 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:16.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:16.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:16.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:16.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:16.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:16.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:16.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:16.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:16.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:16.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:16.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:16.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:16.074 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:16.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:16.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:16.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.076 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:16.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:16.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.077 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:16.078 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:16.078 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.90326341700001, 2023-10-30T06:30:16.078 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:16.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.079 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.079 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.079 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.079 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:16.079 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.080 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.080 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.080 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:16.082 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:16.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.082 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.082 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.083 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:16.084 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:16.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.084 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.084 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.084 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:16.085 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:16.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.086 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.086 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.086 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:16.086 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:16.087 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:16.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.088 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.088 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.088 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:16.088 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.089 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:16.090 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:16.090 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.52935233300002, 2023-10-30T06:30:16.090 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:16.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.091 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.091 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.091 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.091 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:16.091 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.093 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.093 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.093 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:16.093 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:16.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.094 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.094 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.094 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:16.094 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:16.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.096 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.097 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.098 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.098 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:16.098 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:16.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.099 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.101 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:16.101 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:16.101 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.52485363199997, 2023-10-30T06:30:16.101 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:16.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.102 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.102 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.103 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:16.104 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:16.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.105 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.105 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:16.106 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:16.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.106 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.106 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:16.107 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:16.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.107 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.107 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:16.108 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:16.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.109 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.109 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:16.109 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:16.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.111 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.112 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.112 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.112 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:16.112 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:16.113 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.233249693, 2023-10-30T06:30:16.113 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:16.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.113 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.114 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.114 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.114 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:16.114 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.115 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.116 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.116 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:16.116 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:16.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.117 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.118 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.118 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:16.118 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:16.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.118 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.119 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.120 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:16.121 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:16.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.121 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.121 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.121 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:16.122 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:16.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.123 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.124 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:16.124 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:16.124 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.22731237599999, 2023-10-30T06:30:16.124 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:16.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.125 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.125 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:16.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.126 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.126 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:16.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.127 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:16.128 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:16.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:16.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:16.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.129 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.129 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:16.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:16.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.131 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.131 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:16.132 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:16.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.132 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.132 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.132 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:16.133 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:16.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.133 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.133 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.133 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:16.134 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:16.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.134 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.134 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:16.135 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:16.135 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.135 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.135 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.136 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.136 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:16.136 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:16.136 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.22601336100001, 2023-10-30T06:30:16.136 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:16.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.137 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.137 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.138 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:16.138 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.138 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.138 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.139 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:16.140 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:16.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.140 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.140 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:16.142 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:16.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.142 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.142 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.142 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:16.143 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:16.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.143 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.143 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.143 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:16.144 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:16.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.144 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.144 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.144 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:16.145 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:16.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.146 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.146 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.146 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:16.146 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.147 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.148 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:16.148 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:16.148 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 359.12302455299999, 2023-10-30T06:30:16.148 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:16.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.149 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.149 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.149 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.149 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.150 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.151 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.152 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.153 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.154 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.155 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.156 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.157 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.158 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.159 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:16.160 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.85112128600002, 2023-10-30T06:30:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.161 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.163 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.165 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.166 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:16.166 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:16.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.166 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.166 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.167 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:16.167 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:16.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.167 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.168 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:16.168 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:16.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.168 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.169 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:16.169 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:16.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.169 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.169 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.170 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.171 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:16.171 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:16.171 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.171 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.171 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.172 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:16.172 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:16.172 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.84362309400001, 2023-10-30T06:30:16.173 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:16.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.173 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.173 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.174 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.174 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:16.174 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.175 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.175 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.175 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:16.175 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.176 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.176 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.176 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:16.176 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:16.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:16.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.178 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.178 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:16.178 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:16.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.179 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.179 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.179 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:16.179 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:16.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:16.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.181 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.181 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.181 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:16.181 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:16.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.182 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.183 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.183 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:16.183 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:16.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.183 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.184 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.184 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:16.184 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:16.185 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.84296188899998, 2023-10-30T06:30:16.185 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:16.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.185 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.186 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.186 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:16.186 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.188 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.188 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:16.188 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:16.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.190 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.191 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.192 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.193 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.193 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:16.193 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:16.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.193 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.194 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.194 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:16.194 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:16.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.194 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.195 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.195 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:16.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:16.195 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.195 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:16.197 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.83037605099997, 2023-10-30T06:30:16.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:16.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.197 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.198 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:16.199 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:16.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:16.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:16.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.202 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:16.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:16.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:16.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:16.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.204 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:16.205 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:16.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:16.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:16.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.208 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:16.208 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:16.208 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.82428383799999, 2023-10-30T06:30:16.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:16.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.209 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:16.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:16.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:16.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:16.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:16.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:16.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:16.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:16.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:16.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:16.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:16.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:16.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:16.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:16.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:16.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 358.82328849700002, 2023-10-30T06:30:16.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.229 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.230 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:16.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:16.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 351.50061592600002, 2023-10-30T06:30:16.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:16.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:16.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.241 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.242 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 351.467174441, 2023-10-30T06:30:16.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:16.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:16.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:16.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:16.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:16.252 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:16.253 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:16.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:16.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:16.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:16.258 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:16.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:16.261 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:16.259+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:16.361 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:16.359+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:16.427 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:16.428 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:16.428 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:16.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:16.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2023-10-30T06:30:16.480 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:16.479+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:16.544 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:30:16.563 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:16.561+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:16.563 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:16.563 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:16.564 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:16.564 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:16.564 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:16.572 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:16.663 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:16.662+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:16.740 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:16.740 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:16.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:16.741 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:16.741 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:16.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:16.792 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:16.791+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:16.866 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:16.864+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:16.875 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:16.875 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:16.876 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:16.876 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:16.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:16.967 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:16.965+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:17.058 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:17.058 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:17.058 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:17.058 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:17.059 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:17.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:17.068 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:17.067+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:17.178 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:17.176+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:17.195 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:17.195 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:17.196 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:17.196 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:17.196 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.196 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:17.196 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:17.197 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 280.22157088400002, 2023-10-30T06:30:17.197 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:17.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:17.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:17.198 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:17.198 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:17.198 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:17.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:17.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.199 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:17.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.199 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:17.199 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:17.200 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:17.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.200 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.200 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:17.200 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:17.201 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:17.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.201 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:17.202 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:17.202 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:17.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.202 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.203 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:17.203 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:17.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:17.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.204 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:17.204 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:17.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:17.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.205 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.205 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:17.205 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:17.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:17.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:17.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:17.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:17.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:17.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:17.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:17.208 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:17.209 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:17.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:17.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.210 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:17.210 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:17.210 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 100.20107856, 2023-10-30T06:30:17.210 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:17.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:17.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:17.211 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:17.211 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:17.211 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:17.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:17.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.212 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:17.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:17.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:17.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:17.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:17.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:17.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:17.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:17.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:17.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:17.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.216 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:17.216 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:17.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:17.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.218 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:17.218 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:17.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:17.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:17.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:17.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:17.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:17.220 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:17.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:17.221 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:17.221 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:17.221 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:17.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:17.271 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:17.269+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:17.351 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:17.352 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:17.352 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:17.352 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:17.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:17.402 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:17.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:17.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2023-10-30T06:30:17.473 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:17.472+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:17.536 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:17.536 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:17.537 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:17.537 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:17.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:17.574 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:17.572+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:17.674 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:17.675 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:17.675 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:17.676 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:17.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:17.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:17.776 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:17.774+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:17.849 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:17.849 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:17.849 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:17.850 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:17.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:17.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:17.903 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:17.901+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:17.978 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:17.977+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:18.004 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:18.005 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:18.005 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:18.014 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:18.079 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:18.078+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:18.152 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:18.153 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.180 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:18.179+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:18.245 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:18.245 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:18.246 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 364.90587818199998, 2023-10-30T06:30:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.248 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.250 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:18.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:18.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:18.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.265 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.58472299099998, 2023-10-30T06:30:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.266 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.275 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.58047843399999, 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:18.281 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.282 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.283 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:18.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:18.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:18.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:18.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:18.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:18.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:18.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.295 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.296 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.49502000899997, 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.297 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.299 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:18.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:18.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:18.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:18.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:18.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:18.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.311 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.22577679300002, 2023-10-30T06:30:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.312 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:18.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:18.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:18.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:18.325 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.326 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.22494538199999, 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.327 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.328 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:18.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:18.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.224278255, 2023-10-30T06:30:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:18.342 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:18.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:18.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:18.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:18.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.20200418399997, 2023-10-30T06:30:18.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.358 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:18.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:18.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:18.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:18.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:18.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.371 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.20135632799997, 2023-10-30T06:30:18.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.373 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:18.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:18.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:18.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:18.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:18.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:18.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.386 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:18.387 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.20067025499998, 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.388 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:18.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:18.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:18.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:18.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 361.129788287, 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.403 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:18.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:18.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:18.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:18.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:18.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:18.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.417 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.417 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 361.12912432100001, 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.418 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.419 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.419 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:18.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.420 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:18.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:18.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:18.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:18.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:18.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:18.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:18.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:18.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:18.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:18.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:18.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:18.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:18.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:18.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:18.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:18.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:18.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.432 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.432 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 361.10067051200002, 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:18.433 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.434 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:18.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:18.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:18.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:18.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.758904019, 2023-10-30T06:30:18.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:18.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:18.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:18.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:18.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:18.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:18.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.462 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.463 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.75815534899999, 2023-10-30T06:30:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.464 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:18.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:18.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:18.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.75748696599999, 2023-10-30T06:30:18.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.482 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:18.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:18.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:18.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:18.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:18.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:18.496 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.74763629199998, 2023-10-30T06:30:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:18.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:18.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:18.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:18.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:18.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:18.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:18.512 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.74696247700001, 2023-10-30T06:30:18.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:18.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.513 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:18.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:18.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:18.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:18.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.526 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.74631409400001, 2023-10-30T06:30:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.527 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:18.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:18.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:18.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 353.74219762799999, 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:18.542 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:18.543 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:18.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:18.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:18.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:18.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:18.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:18.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:18.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:18.557 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:18.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:18.558 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:18.558 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:30:18.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:30:18.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:18.560 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.561 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.562 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2023-10-30T06:30:18.564 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:18.564 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:30:18.565 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:18.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:18.634 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:18.634 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.675 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:18.673+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:18.736 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:18.736 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.747 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:18.747 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:18.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:18.762 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:18.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:18.836 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:18.894 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:18.892+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:18.940 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:18.940 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:18.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:18.961 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:18.960+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:19.062 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:19.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:19.139 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:19.139 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:19.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:19.191 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:19.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:19.265 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:19.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:19.328 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:19.328 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:19.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:19.366 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:19.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:19.466 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:19.465+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:19.548 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:19.549 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:19.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:19.604 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:19.602+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:19.668 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:19.667+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:19.690 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:19.690 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:19.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:19.750 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:30:19.750 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:19.751 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 106, 2023-10-30T06:30:19.751 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:30:19.751 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:30:19.751 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:30:16.521877+0000", 2023-10-30T06:30:19.751 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:30:19.752 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:30:13.065550+0000", 2023-10-30T06:30:19.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:30:19.752 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5799936, 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:30:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.754 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:30:19.754 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:30:19.754 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:30:19.754 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:30:19.755 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:30:19.755 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:30:19.755 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:30:19.755 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:30:19.755 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:30:19.756 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:30:19.756 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:30:19.756 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:30:19.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.756 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:30:19.757 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:30:19.758 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:30:19.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:30:19.758 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:30:19.758 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:30:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:30:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:30:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:30:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:30:19.759 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:30:19.760 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:30:19.760 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:30:19.760 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:30:19.760 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:30:19.760 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:30:19.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:30:19.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:30:19.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:30:19.761 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:30:19.761 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:30:19.762 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:30:19.762 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:30:19.762 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:30:19.762 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:30:19.762 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:30:19.763 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:30:19.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.763 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:30:19.763 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:30:19.763 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:30:19.764 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:30:19.764 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:30:19.764 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:30:19.764 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:30:19.764 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:30:19.765 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:30:19.765 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:30:19.765 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:30:19.765 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:30:19.765 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:30:19.766 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:30:19.766 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:30:19.766 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:30:19.766 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:30:19.766 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:30:19.767 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:30:19.767 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:30:19.767 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:30:19.767 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:30:19.767 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:30:19.768 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:30:19.768 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:30:19.768 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:30:19.768 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:30:19.768 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.769 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:30:19.769 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:30:19.769 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:30:19.769 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:30:19.769 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:30:19.770 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:30:19.770 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:30:19.770 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:30:19.770 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:30:19.770 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:30:19.771 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:30:19.771 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:30:19.771 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:30:19.771 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:30:19.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.772 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:30:19.772 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:30:19.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.772 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:30:19.772 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.77820587158203125, 2023-10-30T06:30:19.773 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.88999998569488525, 2023-10-30T06:30:19.774 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.77999997138977051 2023-10-30T06:30:19.774 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.774 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:30:19.775 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:30:19.775 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:30:19.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:30:19.775 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:30:19.775 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:30:19.776 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:30:19.776 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:30:19.776 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:30:19.776 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:30:19.776 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:30:19.777 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:30:19.777 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:30:19.777 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:30:19.777 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:30:19.777 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:30:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:30:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:30:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:30:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:30:19.778 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:30:19.779 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:30:19.779 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:30:19.779 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:30:19.779 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:30:19.779 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:30:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:30:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:30:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:30:19.780 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:30:19.781 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:30:19.781 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:30:19.781 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:30:19.781 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:30:19.781 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:30:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:30:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:30:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:30:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:30:19.782 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:30:19.783 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:30:19.783 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:30:19.783 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:30:19.783 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:30:19.783 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:30:19.784 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:30:19.784 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:30:19.784 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:30:19.784 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:30:19.784 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:30:19.785 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:30:19.785 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:30:19.785 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:30:19.785 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:30:19.786 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.786 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:30:19.786 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:30:19.786 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:30:19.786 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:30:19.787 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:30:19.787 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:30:19.787 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:30:19.787 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:30:19.787 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:30:19.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.789 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:30:19.789 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 3.5099999904632568, 2023-10-30T06:30:19.789 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 3.5099999904632568, 2023-10-30T06:30:19.789 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:30:19.789 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 4, 2023-10-30T06:30:19.790 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 4, 2023-10-30T06:30:19.790 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.77820587158203125, 2023-10-30T06:30:19.790 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.88999998569488525, 2023-10-30T06:30:19.790 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.77999997138977051 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.791 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:30:19.792 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:30:19.792 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.792 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.792 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.792 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:19.793 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.794 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.794 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.794 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.794 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.795 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:30:19.795 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.795 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:30:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.797 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.797 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.797 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.797 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:30:19.798 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.798 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.798 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:30:19.799 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.799 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.799 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.799 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.800 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.800 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.800 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:30:19.800 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.801 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.801 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:30:19.801 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.802 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.802 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.802 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.802 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.802 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.803 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.804 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.804 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:30:19.804 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:19.804 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.804 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.805 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:30:19.805 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:30:19.805 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:30:19.805 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:30:19.806 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.806 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.806 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.806 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.807 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:30:19.807 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:30:19.807 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.807 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.808 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.808 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.808 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.808 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.808 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:30:19.809 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:19.809 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.809 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.809 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.809 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.810 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.810 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:30:19.810 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.811 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.811 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:30:19.811 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.811 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.812 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.812 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.812 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.813 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.813 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:30:19.813 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.814 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.814 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:30:19.814 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.814 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.815 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.815 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.815 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.816 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.816 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:30:19.816 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.818 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.818 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.818 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.819 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:30:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.820 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:30:19.821 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:30:19.821 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:30:19.821 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:30:19.821 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.821 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.822 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.822 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.822 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:30:19.823 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:30:19.823 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.823 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.823 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:19.824 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.825 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.825 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.825 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.825 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:30:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.826 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:30:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.828 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.828 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.828 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:30:19.829 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.829 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.829 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.830 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.831 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.831 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:30:19.831 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.832 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.832 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:30:19.832 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.833 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.833 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.833 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.834 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.834 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:30:19.834 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.835 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.836 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:30:19.836 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:30:19.836 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:30:19.836 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.838 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:30:19.838 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:30:19.838 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.838 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.839 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.839 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.839 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.840 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.840 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:19.840 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 104, 2023-10-30T06:30:19.840 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.840 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:30:19.841 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.842 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.842 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:30:19.842 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.843 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.843 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.843 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.843 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.844 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.844 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:30:19.844 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.845 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.845 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:30:19.845 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.846 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.846 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.846 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.847 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.847 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:30:19.847 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.848 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.850 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:30:19.850 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.850 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.851 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:30:19.851 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:19.851 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.851 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.852 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:30:19.852 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:30:19.852 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:30:19.852 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:30:19.853 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.853 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.853 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.853 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.854 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:30:19.854 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:30:19.854 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.854 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.855 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.855 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.855 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.855 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.855 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:19.856 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:19.856 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.856 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.856 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.856 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.857 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.857 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:30:19.857 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.858 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.858 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:30:19.858 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.858 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.859 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.859 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.859 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.860 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:30:19.860 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.861 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.861 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:30:19.861 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.862 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.862 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.862 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.863 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:30:19.863 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.864 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:30:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.866 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.866 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:30:19.866 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:19.867 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.867 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.867 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:30:19.867 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:30:19.868 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:30:19.868 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:30:19.868 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.868 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.868 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.869 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.869 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:30:19.869 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:30:19.870 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.870 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.870 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.870 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.870 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.871 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.871 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:19.871 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 69, 2023-10-30T06:30:19.871 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.871 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.872 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.872 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.872 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.872 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:30:19.873 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.873 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.873 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:30:19.874 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.874 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.874 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.874 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.875 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.875 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.875 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:30:19.875 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.876 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.876 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:30:19.876 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.877 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.877 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.877 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.878 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:30:19.878 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.880 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:30:19.881 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.881 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:30:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:30:19.883 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:30:19.883 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:30:19.883 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:30:19.883 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.883 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.884 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.884 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.884 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:30:19.885 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:30:19.885 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.885 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.885 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.885 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.22564697265625, 2023-10-30T06:30:19.886 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.886 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.886 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:19.886 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:30:19.886 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.887 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.887 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.887 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.887 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.888 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:30:19.889 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.889 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.889 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.889 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.890 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.890 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.890 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:30:19.890 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.891 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.891 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:30:19.891 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.893 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:30:19.893 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.894 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.894 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:30:19.894 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.894 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.896 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:30:19.896 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.896 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.897 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:30:19.897 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:19.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.898 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:30:19.898 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:30:19.898 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:30:19.898 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:30:19.898 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.899 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.899 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.899 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.900 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:30:19.900 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:30:19.900 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:19.900 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:19.900 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:19.901 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:19.901 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:19.901 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:19.901 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:19.901 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:30:19.902 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:19.902 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:19.902 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:19.902 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.903 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.903 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:30:19.903 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.904 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.904 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:30:19.904 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.904 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.904 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.905 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:19.905 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.905 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.906 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:30:19.906 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.906 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.907 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:19.908 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.908 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.908 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:30:19.908 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.909 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.909 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:30:19.909 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.910 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.910 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.910 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:19.910 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:19.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.911 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:19.911 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:30:19.911 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.912 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.913 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:30:19.913 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:30:19.913 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:30:19.913 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:30:19.914 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:19.914 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:19.914 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:19.914 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.914 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.915 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.915 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:30:19.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.915 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:30:19.915 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.916 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.916 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.916 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.916 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.916 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:30:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:30:19.917 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.918 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.918 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.918 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.918 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.918 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:30:19.919 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.919 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:30:19.919 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:30:19.920 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.921 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:30:19.921 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.921 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.922 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.922 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.922 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.922 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:30:19.922 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.923 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.924 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.924 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.924 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:30:19.924 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.925 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:30:19.925 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.925 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.925 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.926 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.926 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.926 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:30:19.926 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.927 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:30:19.927 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.927 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.927 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.928 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.928 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.928 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:30:19.928 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:19.929 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:30:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:19.930 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:30:19.931 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:30:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:30:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.932 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:30:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2023-10-30T06:30:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:30:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:19.933 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 5, 2023-10-30T06:30:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2023-10-30T06:30:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:19.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:30:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:30:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:30:19.935 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:30:19.936 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:30:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:30:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:30:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:19.937 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:30:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:30:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:30:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:30:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:30:19.938 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:30:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:30:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:30:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:30:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:30:19.939 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:30:19.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:19.940 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:19.942 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:19.942 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:19.943 INFO:tasks.thrashosds.thrasher:j is {'epoch': 106, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:30:16.521877+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:30:13.065550+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7782058715820312, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.7799999713897705}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.509999990463257, 'score_stable': 3.509999990463257, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 4, 'raw_score_stable': 4, 'primary_affinity_weighted': 0.7782058715820312, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.7799999713897705}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.22564697265625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [1, 2]}], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:30:19.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:30:19.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:19.945 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:19.940+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:30:20.043 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:20.041+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:20.111 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.112 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:20.143 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:20.142+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:20.244 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:20.243+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:20.286 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.286 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:20.345 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:20.343+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:20.446 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:20.444+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:20.463 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.463 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:20.546 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:30:20.547 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:30:20.548 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:20.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:20.559 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-10-30T06:30:19.878412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224316,"kb_used_data":6400,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750404,"statfs":{"total":773094113280,"available":772864413696,"internally_reserved":0,"allocated":6553600,"data_stored":3702566,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":83,"apply_latency_ms":83,"commit_latency_ns":83000000,"apply_latency_ns":83000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.441691"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-30T06:30:16.969442+0000","last_change":"2023-10-30T06:30:14.747570+0000","last_active":"2023-10-30T06:30:16.969442+0000","last_peered":"2023-10-30T06:30:16.969442+0000","last_clean":"2023-10-30T06:30:16.969442+0000","last_became_active":"2023-10-30T06:30:14.741239+0000","last_became_peered":"2023-10-30T06:30:14.741239+0000","last_unstale":"2023-10-30T06:30:16.969442+0000","last_undegraded":"2023-10-30T06:30:16.969442+0000","last_fullsized":"2023-10-30T06:30:16.969442+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:58:53.403617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-30T06:30:13.531195+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:30:13.531195+0000","last_peered":"2023-10-30T06:30:13.531195+0000","last_clean":"2023-10-30T06:30:13.531195+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:30:13.531195+0000","last_undegraded":"2023-10-30T06:30:13.531195+0000","last_fullsized":"2023-10-30T06:30:13.531195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-30T06:30:14.613160+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:30:14.613160+0000","last_peered":"2023-10-30T06:30:14.613160+0000","last_clean":"2023-10-30T06:30:14.613160+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:30:14.613160+0000","last_undegraded":"2023-10-30T06:30:14.613160+0000","last_fullsized":"2023-10-30T06:30:14.613160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-30T06:30:13.461148+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:30:13.461148+0000","last_peered":"2023-10-30T06:30:13.461148+0000","last_clean":"2023-10-30T06:30:13.461148+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:30:13.461148+0000","last_undegraded":"2023-10-30T06:30:13.461148+0000","last_fullsized":"2023-10-30T06:30:13.461148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-30T06:30:16.962055+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:30:16.962055+0000","last_peered":"2023-10-30T06:30:16.962055+0000","last_clean":"2023-10-30T06:30:16.962055+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:30:16.962055+0000","last_undegraded":"2023-10-30T06:30:16.962055+0000","last_fullsized":"2023-10-30T06:30:16.962055+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":92,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-30T06:30:13.525593+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:30:13.525593+0000","last_peered":"2023-10-30T06:30:13.525593+0000","last_clean":"2023-10-30T06:30:13.525593+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:30:13.525593+0000","last_undegraded":"2023-10-30T06:30:13.525593+0000","last_fullsized":"2023-10-30T06:30:13.525593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-30T06:30:16.960132+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:30:16.960132+0000","last_peered":"2023-10-30T06:30:16.960132+0000","last_clean":"2023-10-30T06:30:16.960132+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:30:16.960132+0000","last_undegraded":"2023-10-30T06:30:16.960132+0000","last_fullsized":"2023-10-30T06:30:16.960132+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-30T06:30:16.967193+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:30:16.967193+0000","last_peered":"2023-10-30T06:30:16.967193+0000","last_clean":"2023-10-30T06:30:16.967193+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:30:16.967193+0000","last_undegraded":"2023-10-30T06:30:16.967193+0000","last_fullsized":"2023-10-30T06:30:16.967193+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":199,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-10-30T06:30:14.609070+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:30:14.609070+0000","last_peered":"2023-10-30T06:30:14.609070+0000","last_clean":"2023-10-30T06:30:14.609070+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:30:14.609070+0000","last_undegraded":"2023-10-30T06:30:14.609070+0000","last_fullsized":"2023-10-30T06:30:14.609070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411405,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":708608,"data_stored":698941,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":10.599,"5min":10.430999999999999,"15min":10.382999999999999},"min":{"1min":4.9269999999999996,"5min":4.7409999999999997,"15min":4.7409999999999997},"max":{"1min":51.225000000000001,"5min":51.225000000000001,"15min":51.225000000000001},"last":8.7880000000000003},{"interface":"front","average":{"1min":10.962,"5min":11.484999999999999,"15min":11.882999999999999},"min":{"1min":5.407,"5min":5.3739999999999997,"15min":5.3739999999999997},"max":{"1min":54.011000000000003,"5min":54.011000000000003,"15min":54.011000000000003},"last":7.9130000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":11.68,"5min":12.036,"15min":13.210000000000001},"min":{"1min":5.3940000000000001,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":31.373000000000001,"5min":52.552,"15min":52.552},"last":19.574000000000002},{"interface":"front","average":{"1min":15.313000000000001,"5min":14.664999999999999,"15min":14.986000000000001},"min":{"1min":7.8899999999999997,"5min":7.1589999999999998,"15min":7.1589999999999998},"max":{"1min":35.033999999999999,"5min":55.585999999999999,"15min":55.585999999999999},"last":20.882999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":14.407999999999999,"5min":16.832999999999998,"15min":21.071000000000002},"min":{"1min":9.0489999999999995,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":23.529,"5min":118.45699999999999,"15min":118.45699999999999},"last":11.505000000000001},{"interface":"front","average":{"1min":13.064,"5min":15.606999999999999,"15min":20.25},"min":{"1min":7.5819999999999999,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":26.908999999999999,"5min":124.075,"15min":124.075},"last":9.4079999999999995}]},{"osd":4,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":12.913,"5min":13.177,"15min":13.509},"min":{"1min":7.1200000000000001,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":33.033000000000001,"5min":51.649000000000001,"15min":51.649000000000001},"last":12.121},{"interface":"front","average":{"1min":16.995999999999999,"5min":17.140000000000001,"15min":17.712},"min":{"1min":10.973000000000001,"5min":7.9400000000000004,"15min":7.9400000000000004},"max":{"1min":27.577999999999999,"5min":44.973999999999997,"15min":44.973999999999997},"last":10.435}]},{"osd":5,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":17.241,"5min":17.216000000000001,"15min":19.773},"min":{"1min":8.984,"5min":7.351,"15min":7.351},"max":{"1min":36.764000000000003,"5min":123.754,"15min":123.754},"last":13.911},{"interface":"front","average":{"1min":15.685,"5min":16.792999999999999,"15min":20.984000000000002},"min":{"1min":9.3840000000000003,"5min":7.2199999999999998,"15min":7.2199999999999998},"max":{"1min":35.218000000000004,"5min":122.72499999999999,"15min":122.72499999999999},"last":6.8520000000000003}]},{"osd":6,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":17.279,"5min":18.350999999999999,"15min":20.420999999999999},"min":{"1min":9.7170000000000005,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":37.066000000000003,"5min":66.400999999999996,"15min":66.400999999999996},"last":15.140000000000001},{"interface":"front","average":{"1min":16.552,"5min":17.384,"15min":19.033999999999999},"min":{"1min":11.015000000000001,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":35.822000000000003,"5min":53.149000000000001,"15min":53.149000000000001},"last":15.858000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":17.353999999999999,"5min":18.899000000000001,"15min":20.641999999999999},"min":{"1min":11.714,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.972999999999999,"5min":51.375,"15min":51.375},"last":12.727},{"interface":"front","average":{"1min":18.661999999999999,"5min":19.818999999999999,"15min":21.003},"min":{"1min":11.135999999999999,"5min":8.9930000000000003,"15min":8.9930000000000003},"max":{"1min":38.521000000000001,"5min":69.991,"15min":69.991},"last":14.532}]}]},{"osd":6,"up_from":16,"seq":68719476814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":856064,"data_stored":384114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":10.247,"5min":11.227,"15min":13.545999999999999},"min":{"1min":5.2380000000000004,"5min":4.7779999999999996,"15min":4.7779999999999996},"max":{"1min":22.741,"5min":94.343000000000004,"15min":94.343000000000004},"last":8.0250000000000004},{"interface":"front","average":{"1min":9.6560000000000006,"5min":11.369999999999999,"15min":14.292},"min":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"max":{"1min":21.756,"5min":122.054,"15min":122.054},"last":12.220000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":12.568,"5min":10.816000000000001,"15min":10.816000000000001},"min":{"1min":6.6779999999999999,"5min":5.5590000000000002,"15min":5.5590000000000002},"max":{"1min":24.128,"5min":24.128,"15min":24.128},"last":9.5950000000000006},{"interface":"front","average":{"1min":13.832000000000001,"5min":17.963000000000001,"15min":28.562999999999999},"min":{"1min":6.7060000000000004,"5min":5.9269999999999996,"15min":5.9269999999999996},"max":{"1min":21.541,"5min":387.15600000000001,"15min":387.15600000000001},"last":14.036}]},{"osd":2,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":15.548999999999999,"5min":14.403,"15min":18.105},"min":{"1min":6.5999999999999996,"5min":6.4359999999999999,"15min":6.4359999999999999},"max":{"1min":72.988,"5min":156.90299999999999,"15min":156.90299999999999},"last":10.926},{"interface":"front","average":{"1min":20.797000000000001,"5min":17.241,"15min":19.469000000000001},"min":{"1min":11.146000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":75.004000000000005,"5min":143.578,"15min":143.578},"last":16.734999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":15.782,"5min":17.678999999999998,"15min":20.535},"min":{"1min":9.8149999999999995,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":38.576999999999998,"5min":141.18799999999999,"15min":141.18799999999999},"last":8.875},{"interface":"front","average":{"1min":17.527000000000001,"5min":20.335000000000001,"15min":23.725000000000001},"min":{"1min":9.2799999999999994,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":31.260000000000002,"5min":161.32900000000001,"15min":161.32900000000001},"last":13.362}]},{"osd":4,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":10.33,"5min":12.446999999999999,"15min":14.179},"min":{"1min":7.4690000000000003,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":16.370999999999999,"5min":74.079999999999998,"15min":74.079999999999998},"last":15.971},{"interface":"front","average":{"1min":15.141,"5min":14.890000000000001,"15min":15.430999999999999},"min":{"1min":8.6649999999999991,"5min":7.6379999999999999,"15min":7.6379999999999999},"max":{"1min":25.407,"5min":58.613999999999997,"15min":58.613999999999997},"last":11.502000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":12.872,"5min":14.289,"15min":16.739999999999998},"min":{"1min":7.5599999999999996,"5min":7.383,"15min":7.383},"max":{"1min":18.826000000000001,"5min":71.757000000000005,"15min":71.757000000000005},"last":48.072000000000003},{"interface":"front","average":{"1min":17.902999999999999,"5min":19.035,"15min":20.853000000000002},"min":{"1min":12.256,"5min":9.532,"15min":9.532},"max":{"1min":25.789000000000001,"5min":80.119,"15min":80.119},"last":49.463999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":14.842000000000001,"5min":16.437000000000001,"15min":18.638999999999999},"min":{"1min":10.099,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":19.539000000000001,"5min":81.247,"15min":81.247},"last":14.775},{"interface":"front","average":{"1min":18.655999999999999,"5min":17.079000000000001,"15min":18.417999999999999},"min":{"1min":11.757,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":28.073,"5min":70.989000000000004,"15min":70.989000000000004},"last":17.626000000000001}]}]},{"osd":3,"up_from":16,"seq":68719476814,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":856064,"data_stored":384114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":10.946999999999999,"5min":10.847,"15min":10.989000000000001},"min":{"1min":5.7149999999999999,"5min":4.2679999999999998,"15min":4.2679999999999998},"max":{"1min":25.015000000000001,"5min":48.792000000000002,"15min":48.792000000000002},"last":7.0529999999999999},{"interface":"front","average":{"1min":10.768000000000001,"5min":10.904,"15min":10.473000000000001},"min":{"1min":6.6429999999999998,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":22.754999999999999,"5min":28.616,"15min":28.616},"last":6.0700000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":14.896000000000001,"5min":17.207999999999998,"15min":23.620000000000001},"min":{"1min":6.2939999999999996,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":30.465,"5min":165.76400000000001,"15min":165.76400000000001},"last":12.076000000000001},{"interface":"front","average":{"1min":13.653,"5min":17.068999999999999,"15min":24.408999999999999},"min":{"1min":6.2880000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":25.407,"5min":190.34200000000001,"15min":190.34200000000001},"last":15.367000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":17.224,"5min":16.513999999999999,"15min":17.789999999999999},"min":{"1min":7.8890000000000002,"5min":7.8890000000000002,"15min":7.8890000000000002},"max":{"1min":42.881999999999998,"5min":42.881999999999998,"15min":42.881999999999998},"last":16.716000000000001},{"interface":"front","average":{"1min":18.937999999999999,"5min":16.332000000000001,"15min":16.91},"min":{"1min":8.7319999999999993,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":71.721000000000004,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.409000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":18.155999999999999,"5min":18.058,"15min":19.765999999999998},"min":{"1min":6.6040000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":34.954999999999998,"5min":44.091000000000001,"15min":44.091000000000001},"last":17.501999999999999},{"interface":"front","average":{"1min":12.702,"5min":11.254,"15min":11.494},"min":{"1min":7.0019999999999998,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":23.815000000000001,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.9570000000000007}]},{"osd":5,"last update":"Mon Oct 30 06:29:25 2023","interfaces":[{"interface":"back","average":{"1min":19.504000000000001,"5min":16.474,"15min":18.597000000000001},"min":{"1min":10.821,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":57.375999999999998,"5min":57.375999999999998,"15min":57.375999999999998},"last":26.584},{"interface":"front","average":{"1min":13.99,"5min":15.926,"15min":19.173999999999999},"min":{"1min":8.548,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.248000000000001,"5min":58.576000000000001,"15min":58.576000000000001},"last":27.321000000000002}]},{"osd":6,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":14.66,"5min":19.367999999999999,"15min":25.881},"min":{"1min":7.0469999999999997,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":25.617999999999999,"5min":169.05199999999999,"15min":169.05199999999999},"last":9.9049999999999994},{"interface":"front","average":{"1min":18.199000000000002,"5min":15.321999999999999,"15min":15.436},"min":{"1min":10.561,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":27.931000000000001,"5min":40.128999999999998,"15min":40.128999999999998},"last":11.409000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:29:20 2023","interfaces":[{"interface":"back","average":{"1min":23.361000000000001,"5min":19.417999999999999,"15min":18.463000000000001},"min":{"1min":13.488,"5min":8.5950000000000006,"15min":8.5950000000000006},"max":{"1min":64.313999999999993,"5min":64.313999999999993,"15min":64.313999999999993},"last":14.723000000000001},{"interface":"front","average":{"1min":22.187999999999999,"5min":22.195,"15min":27.140999999999998},"min":{"1min":9.5990000000000002,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":55.331000000000003,"5min":170.232,"15min":170.232},"last":18.303999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":856064,"data_stored":384114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:15 2023","interfaces":[{"interface":"back","average":{"1min":15.013999999999999,"5min":14.85,"15min":23.489000000000001},"min":{"1min":5.1950000000000003,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":91.290999999999997,"5min":258.66199999999998,"15min":258.66199999999998},"last":20.334},{"interface":"front","average":{"1min":15.629,"5min":14.257,"15min":20.614000000000001},"min":{"1min":6.1219999999999999,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":92.879999999999995,"5min":203.38499999999999,"15min":203.38499999999999},"last":10.965999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":13.436999999999999,"15min":15.353},"min":{"1min":6.7359999999999998,"5min":6.1230000000000002,"15min":6.1230000000000002},"max":{"1min":31.689,"5min":45.936999999999998,"15min":45.936999999999998},"last":11.526999999999999},{"interface":"front","average":{"1min":13.317,"5min":12.864000000000001,"15min":14.18},"min":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":35.034999999999997,"5min":37.390999999999998,"15min":37.390999999999998},"last":18.061}]},{"osd":3,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":9.2249999999999996,"5min":12.557,"15min":17.068999999999999},"min":{"1min":6.1950000000000003,"5min":6.1950000000000003,"15min":6.1950000000000003},"max":{"1min":14.919,"5min":183.38900000000001,"15min":183.38900000000001},"last":10.180999999999999},{"interface":"front","average":{"1min":13.318,"5min":16.562999999999999,"15min":20.899000000000001},"min":{"1min":8.4860000000000007,"5min":6.8810000000000002,"15min":6.8810000000000002},"max":{"1min":19.129999999999999,"5min":189.374,"15min":189.374},"last":24.312000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":12.387,"5min":15.1,"15min":19.672000000000001},"min":{"1min":6.9210000000000003,"5min":6.9210000000000003,"15min":6.9210000000000003},"max":{"1min":31.632000000000001,"5min":186.01400000000001,"15min":186.01400000000001},"last":13.134},{"interface":"front","average":{"1min":16.507000000000001,"5min":15.659000000000001,"15min":18.629000000000001},"min":{"1min":6.7679999999999998,"5min":6.2400000000000002,"15min":6.2400000000000002},"max":{"1min":33.311,"5min":167.67599999999999,"15min":167.67599999999999},"last":19.393999999999998}]},{"osd":5,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":15.792,"5min":19.132000000000001,"15min":24.75},"min":{"1min":8.843,"5min":8.3040000000000003,"15min":8.3040000000000003},"max":{"1min":25.699999999999999,"5min":215.80500000000001,"15min":215.80500000000001},"last":25.954000000000001},{"interface":"front","average":{"1min":16.815999999999999,"5min":18.433,"15min":24.364999999999998},"min":{"1min":9.1170000000000009,"5min":8.6989999999999998,"15min":8.6989999999999998},"max":{"1min":25.248999999999999,"5min":187.863,"15min":187.863},"last":21.847000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":14.090999999999999,"5min":17.736999999999998,"15min":24.087},"min":{"1min":7.7969999999999997,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":21.202000000000002,"5min":228.81200000000001,"15min":228.81200000000001},"last":27.274000000000001},{"interface":"front","average":{"1min":17.085000000000001,"5min":21.123999999999999,"15min":27.274000000000001},"min":{"1min":10.103,"5min":9.3710000000000004,"15min":9.3710000000000004},"max":{"1min":30.364999999999998,"5min":275.92200000000003,"15min":275.92200000000003},"last":13.896000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:29:19 2023","interfaces":[{"interface":"back","average":{"1min":19.053000000000001,"5min":20.100000000000001,"15min":24.850999999999999},"min":{"1min":15.256,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":29.216999999999999,"5min":170.43600000000001,"15min":170.43600000000001},"last":23.096},{"interface":"front","average":{"1min":16.504000000000001,"5min":18.454999999999998,"15min":21.843},"min":{"1min":10.675000000000001,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":26.312999999999999,"5min":157.00200000000001,"15min":157.00200000000001},"last":14.962}]}]},{"osd":0,"up_from":16,"seq":68719476813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":671744,"data_stored":301101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":11.067,"5min":10.859999999999999,"15min":10.75},"min":{"1min":4.7290000000000001,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":50.189,"5min":56.704000000000001,"15min":56.704000000000001},"last":8.2089999999999996},{"interface":"front","average":{"1min":11.630000000000001,"5min":11.779,"15min":11.865},"min":{"1min":4.5949999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":47.033000000000001,"5min":57.805,"15min":57.805},"last":7.4089999999999998}]},{"osd":2,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":10.926,"5min":12.172000000000001,"15min":17.425000000000001},"min":{"1min":5.6660000000000004,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":32.957000000000001,"5min":127.95399999999999,"15min":127.95399999999999},"last":16.690000000000001},{"interface":"front","average":{"1min":13.205,"5min":13.039999999999999,"15min":14.260999999999999},"min":{"1min":7.1749999999999998,"5min":6.8070000000000004,"15min":6.8070000000000004},"max":{"1min":33.015000000000001,"5min":35.834000000000003,"15min":35.834000000000003},"last":9.7119999999999997}]},{"osd":3,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":14.755000000000001,"5min":15.32,"15min":20.206},"min":{"1min":7.7949999999999999,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":29.050000000000001,"5min":99.790999999999997,"15min":99.790999999999997},"last":13.303000000000001},{"interface":"front","average":{"1min":16.850999999999999,"5min":15.704000000000001,"15min":16.378},"min":{"1min":6.516,"5min":6.516,"15min":6.516},"max":{"1min":32.545000000000002,"5min":43.860999999999997,"15min":43.860999999999997},"last":9.0540000000000003}]},{"osd":4,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":14.673,"5min":13.991,"15min":15.763},"min":{"1min":7.1559999999999997,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":27.417999999999999,"5min":51.802,"15min":51.802},"last":12.419},{"interface":"front","average":{"1min":15.798,"5min":17.853000000000002,"15min":22.867000000000001},"min":{"1min":8.5649999999999995,"5min":7.6689999999999996,"15min":7.6689999999999996},"max":{"1min":27.170000000000002,"5min":103.705,"15min":103.705},"last":30.931000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":14.724,"5min":15.032999999999999,"15min":17.702999999999999},"min":{"1min":8.8439999999999994,"5min":7.8330000000000002,"15min":7.8330000000000002},"max":{"1min":34.07,"5min":43.847000000000001,"15min":43.847000000000001},"last":14.27},{"interface":"front","average":{"1min":17.428999999999998,"5min":19.661999999999999,"15min":25.452000000000002},"min":{"1min":10.911,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":30.960000000000001,"5min":98.707999999999998,"15min":98.707999999999998},"last":31.928000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":18.483000000000001,"5min":18.606999999999999,"15min":19.870999999999999},"min":{"1min":9.3829999999999991,"5min":8.6869999999999994,"15min":8.6869999999999994},"max":{"1min":36.777999999999999,"5min":55.631,"15min":55.631},"last":11.144},{"interface":"front","average":{"1min":17.218,"5min":18.196000000000002,"15min":20.931999999999999},"min":{"1min":10.656000000000001,"5min":9.7159999999999993,"15min":9.7159999999999993},"max":{"1min":35.923999999999999,"5min":54.656999999999996,"15min":54.656999999999996},"last":14.891}]},{"osd":7,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":19.928999999999998,"5min":19.402000000000001,"15min":21.216000000000001},"min":{"1min":9.9700000000000006,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":41.447000000000003,"5min":53.314,"15min":53.314},"last":11.718999999999999},{"interface":"front","average":{"1min":19.579000000000001,"5min":19.341000000000001,"15min":21.969000000000001},"min":{"1min":12.76,"5min":10.589,"15min":10.589},"max":{"1min":47.578000000000003,"5min":47.578000000000003,"15min":47.578000000000003},"last":17.655000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":856064,"data_stored":384114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:27 2023","interfaces":[{"interface":"back","average":{"1min":9.6029999999999998,"5min":11.284000000000001,"15min":14.657999999999999},"min":{"1min":4.9219999999999997,"5min":4.4249999999999998,"15min":4.4249999999999998},"max":{"1min":27.265000000000001,"5min":131.477,"15min":131.477},"last":14.313000000000001},{"interface":"front","average":{"1min":10.817,"5min":12.044,"15min":15.004},"min":{"1min":6.2060000000000004,"5min":5.6340000000000003,"15min":5.6340000000000003},"max":{"1min":29.16,"5min":129.386,"15min":129.386},"last":6.5730000000000004}]},{"osd":1,"last update":"Mon Oct 30 06:29:27 2023","interfaces":[{"interface":"back","average":{"1min":10.887,"5min":10.916,"15min":10.968},"min":{"1min":6.0599999999999996,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":15.625,"5min":25.774000000000001,"15min":25.774000000000001},"last":43.533999999999999},{"interface":"front","average":{"1min":15.457000000000001,"5min":16.608000000000001,"15min":18.527999999999999},"min":{"1min":10.135,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":20.248000000000001,"5min":91.295000000000002,"15min":91.295000000000002},"last":44.848999999999997}]},{"osd":2,"last update":"Mon Oct 30 06:29:27 2023","interfaces":[{"interface":"back","average":{"1min":10.173999999999999,"5min":13.137,"15min":16.524999999999999},"min":{"1min":6.7880000000000003,"5min":6.0830000000000002,"15min":6.0830000000000002},"max":{"1min":17.588000000000001,"5min":134.21199999999999,"15min":134.21199999999999},"last":7.5439999999999996},{"interface":"front","average":{"1min":13.798999999999999,"5min":13.836,"15min":14.368},"min":{"1min":7.6139999999999999,"5min":5.992,"15min":5.992},"max":{"1min":21.364999999999998,"5min":50.450000000000003,"15min":50.450000000000003},"last":13.036}]},{"osd":3,"last update":"Mon Oct 30 06:29:27 2023","interfaces":[{"interface":"back","average":{"1min":17.891999999999999,"5min":16.033999999999999,"15min":19.123999999999999},"min":{"1min":6.9610000000000003,"5min":6.758,"15min":6.758},"max":{"1min":67.840000000000003,"5min":132.56100000000001,"15min":132.56100000000001},"last":8.6159999999999997},{"interface":"front","average":{"1min":18.541,"5min":17.234999999999999,"15min":19.870000000000001},"min":{"1min":10.204000000000001,"5min":6.5780000000000003,"15min":6.5780000000000003},"max":{"1min":66.106999999999999,"5min":124.684,"15min":124.684},"last":10.457000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:29:23 2023","interfaces":[{"interface":"back","average":{"1min":17.792999999999999,"5min":14.036,"15min":13.936999999999999},"min":{"1min":9.5109999999999992,"5min":6.6050000000000004,"15min":6.6050000000000004},"max":{"1min":36.552,"5min":36.552,"15min":36.552},"last":13.708},{"interface":"front","average":{"1min":17.352,"5min":16.396999999999998,"15min":15.867000000000001},"min":{"1min":10.849,"5min":8.3089999999999993,"15min":8.3089999999999993},"max":{"1min":36.908999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":11.089}]},{"osd":5,"last update":"Mon Oct 30 06:29:23 2023","interfaces":[{"interface":"back","average":{"1min":11.788,"5min":14.522,"15min":18.657},"min":{"1min":8.0890000000000004,"5min":6.9550000000000001,"15min":6.9550000000000001},"max":{"1min":19.085999999999999,"5min":125.792,"15min":125.792},"last":11.792},{"interface":"front","average":{"1min":17.183,"5min":16.891999999999999,"15min":16.968},"min":{"1min":11.518000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":29.338000000000001,"5min":29.338000000000001,"15min":29.338000000000001},"last":15.691000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:29:27 2023","interfaces":[{"interface":"back","average":{"1min":18.698,"5min":20.265000000000001,"15min":28.199999999999999},"min":{"1min":11.074999999999999,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":33.497,"5min":327.29599999999999,"15min":327.29599999999999},"last":9.4559999999999995},{"interface":"front","average":{"1min":12.895,"5min":18.172000000000001,"15min":28.670000000000002},"min":{"1min":8.7959999999999994,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":20.809000000000001,"5min":323.82400000000001,"15min":323.82400000000001},"last":14.946999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":892928,"data_stored":781954,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":12.832000000000001,"5min":11.244,"15min":13.805999999999999},"min":{"1min":5.3680000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":45.569000000000003,"5min":68.513999999999996,"15min":68.513999999999996},"last":5.8289999999999997},{"interface":"front","average":{"1min":10.807,"5min":11.613,"15min":15.231},"min":{"1min":5.9560000000000004,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":42.341999999999999,"5min":88.816000000000003,"15min":88.816000000000003},"last":10.022}]},{"osd":1,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":14.840999999999999,"5min":14.497999999999999,"15min":16.588999999999999},"min":{"1min":6.9870000000000001,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":38.808,"5min":87.515000000000001,"15min":87.515000000000001},"last":9.5229999999999997},{"interface":"front","average":{"1min":15.109,"5min":14.869999999999999,"15min":16.965},"min":{"1min":7.1449999999999996,"5min":6.21,"15min":6.21},"max":{"1min":43.549999999999997,"5min":85.832999999999998,"15min":85.832999999999998},"last":14.66}]},{"osd":2,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":14.478999999999999,"5min":15.675000000000001,"15min":17.466999999999999},"min":{"1min":7.9989999999999997,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":25.742000000000001,"5min":113.943,"15min":113.943},"last":15.43},{"interface":"front","average":{"1min":10.914,"5min":11.975,"15min":14.305},"min":{"1min":6.5890000000000004,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":18.475000000000001,"5min":80.004000000000005,"15min":80.004000000000005},"last":7.0030000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":16.986999999999998,"5min":17.102,"15min":20.260999999999999},"min":{"1min":7.9969999999999999,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":36.628,"5min":87.866,"15min":87.866},"last":16.187000000000001},{"interface":"front","average":{"1min":16.641999999999999,"5min":18.530999999999999,"15min":22.145},"min":{"1min":7.8390000000000004,"5min":7.8390000000000004,"15min":7.8390000000000004},"max":{"1min":29.427,"5min":91.183000000000007,"15min":91.183000000000007},"last":11.268000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:29:16 2023","interfaces":[{"interface":"back","average":{"1min":15.308999999999999,"5min":12.913,"15min":12.795999999999999},"min":{"1min":6.9619999999999997,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":33.393999999999998,"5min":33.393999999999998,"15min":33.393999999999998},"last":17.109000000000002},{"interface":"front","average":{"1min":16.463999999999999,"5min":16.106000000000002,"15min":16.826000000000001},"min":{"1min":7.5380000000000003,"5min":7.2279999999999998,"15min":7.2279999999999998},"max":{"1min":34.558999999999997,"5min":59.609000000000002,"15min":59.609000000000002},"last":8.9459999999999997}]},{"osd":6,"last update":"Mon Oct 30 06:29:21 2023","interfaces":[{"interface":"back","average":{"1min":17.443000000000001,"5min":15.285,"15min":15.843},"min":{"1min":8.9239999999999995,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":33.962000000000003,"5min":66.765000000000001,"15min":66.765000000000001},"last":10.664},{"interface":"front","average":{"1min":20.413,"5min":19.094000000000001,"15min":20.536000000000001},"min":{"1min":7.8220000000000001,"5min":7.8220000000000001,"15min":7.8220000000000001},"max":{"1min":43.348999999999997,"5min":71.771000000000001,"15min":71.771000000000001},"last":13.696999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:29:16 2023","interfaces":[{"interface":"back","average":{"1min":17.719000000000001,"5min":17.838000000000001,"15min":18.710999999999999},"min":{"1min":8.7059999999999995,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":35.192,"5min":64.677000000000007,"15min":64.677000000000007},"last":17.907},{"interface":"front","average":{"1min":17.632999999999999,"5min":16.594000000000001,"15min":17.785},"min":{"1min":11.250999999999999,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.702999999999999,"5min":42.531999999999996,"15min":42.531999999999996},"last":12.657999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":856064,"data_stored":384114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":9.6739999999999995,"5min":9.6890000000000001,"15min":9.4359999999999999},"min":{"1min":4.8710000000000004,"5min":4.5410000000000004,"15min":4.5410000000000004},"max":{"1min":17.341999999999999,"5min":26.984999999999999,"15min":26.984999999999999},"last":9.4410000000000007},{"interface":"front","average":{"1min":11.183999999999999,"5min":10.786,"15min":10.634},"min":{"1min":5.1210000000000004,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":23.225999999999999,"5min":25.934999999999999,"15min":25.934999999999999},"last":8.1750000000000007}]},{"osd":1,"last update":"Mon Oct 30 06:29:23 2023","interfaces":[{"interface":"back","average":{"1min":11.596,"5min":12.396000000000001,"15min":11.779},"min":{"1min":6.3090000000000002,"5min":6.2960000000000003,"15min":6.2960000000000003},"max":{"1min":21.838999999999999,"5min":25.122,"15min":25.122},"last":11.52},{"interface":"front","average":{"1min":12.779999999999999,"5min":13.406000000000001,"15min":13.904},"min":{"1min":7.476,"5min":6.609,"15min":6.609},"max":{"1min":23.327000000000002,"5min":30.751999999999999,"15min":30.751999999999999},"last":20.141999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":16.321000000000002,"5min":14.069000000000001,"15min":12.667999999999999},"min":{"1min":11.461,"5min":6.6200000000000001,"15min":6.6200000000000001},"max":{"1min":26.809000000000001,"5min":26.809000000000001,"15min":26.809000000000001},"last":12.250999999999999},{"interface":"front","average":{"1min":15.641999999999999,"5min":15.099,"15min":15.550000000000001},"min":{"1min":7.907,"5min":7.907,"15min":7.907},"max":{"1min":34.122999999999998,"5min":34.122999999999998,"15min":34.122999999999998},"last":13.872999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":10.058,"5min":10.444000000000001,"15min":10.991},"min":{"1min":6.6559999999999997,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":19.687999999999999,"5min":66.722999999999999,"15min":66.722999999999999},"last":10.176},{"interface":"front","average":{"1min":15.544,"5min":16.067,"15min":16.065999999999999},"min":{"1min":10.949,"5min":8.1050000000000004,"15min":8.1050000000000004},"max":{"1min":21.184000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":15.742000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":15.662000000000001,"5min":17.393000000000001,"15min":18.866},"min":{"1min":9.843,"5min":6.4950000000000001,"15min":6.4950000000000001},"max":{"1min":35.369999999999997,"5min":64.474999999999994,"15min":64.474999999999994},"last":17.57},{"interface":"front","average":{"1min":13.494,"5min":14.135,"15min":15.519},"min":{"1min":8.5039999999999996,"5min":5.7770000000000001,"15min":5.7770000000000001},"max":{"1min":37.034999999999997,"5min":75.980000000000004,"15min":75.980000000000004},"last":7.1920000000000002}]},{"osd":6,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":17.667999999999999,"5min":18.321999999999999,"15min":18.626000000000001},"min":{"1min":13.456,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":25.190000000000001,"5min":38.712000000000003,"15min":38.712000000000003},"last":18.533999999999999},{"interface":"front","average":{"1min":13.731999999999999,"5min":13.539,"15min":13.863},"min":{"1min":8.9390000000000001,"5min":7.343,"15min":7.343},"max":{"1min":20.219999999999999,"5min":38.668999999999997,"15min":38.668999999999997},"last":10.773999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:29:18 2023","interfaces":[{"interface":"back","average":{"1min":16.664999999999999,"5min":17.042000000000002,"15min":17.919},"min":{"1min":8.7590000000000003,"5min":3.5579999999999998,"15min":3.5579999999999998},"max":{"1min":24.344999999999999,"5min":74.977999999999994,"15min":74.977999999999994},"last":14.795999999999999},{"interface":"front","average":{"1min":17.577999999999999,"5min":19.393999999999998,"15min":21.367999999999999},"min":{"1min":9.9429999999999996,"5min":5.0060000000000002,"15min":5.0060000000000002},"max":{"1min":24.408000000000001,"5min":83.138000000000005,"15min":83.138000000000005},"last":19.366}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:30:20.561 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [1, 2, 3, 4] 2023-10-30T06:30:20.562 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.1', '1', '2', '3', '4'] 2023-10-30T06:30:20.562 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.1 1 2 3 4 2023-10-30T06:30:20.601 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.601 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:20.648 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:20.646+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:20.743 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.744 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.748 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:20.747+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:20.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:20.850 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:20.848+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:20.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:20.881 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:20.882 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:20.891 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:20.950 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:20.949+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:21.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2023-10-30T06:30:21.063 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:21.064 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:21.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:21.091 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:21.089+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:21.105 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.151 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:21.150+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:21.194 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:21.194 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.200 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:21.201 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:21.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:21.252 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:21.250+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:21.282 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:21.283 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.342 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:21.342 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:21.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:21.353 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:21.351+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:21.372 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:21.372 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.459 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:21.459 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.468 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:21.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:21.483 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:21.483 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:21.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:21.546 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.577 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:21.575+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:21.633 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:21.633 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.655 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:21.653+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:21.673 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:21.673 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:21.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:21.730 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:21.731 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:21.755 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:21.753+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:21.811 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:21.812 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:21.812 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:21.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:21.856 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:21.855+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:21.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:30:21.991 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:21.991 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:21.992 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:21.992 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:21.992 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:21.990+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:22.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:22.058 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:22.057+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:22.159 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:22.158+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:22.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:22.171 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:22.172 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:22.172 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:22.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:22.260 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:22.259+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:22.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:22.348 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:22.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:22.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:22.363 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:22.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:22.386 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.1 pg_upmap_items mapping to [1->2,3->4] 2023-10-30T06:30:22.464 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:22.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:22.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2023-10-30T06:30:22.512 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:22.512 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:22.513 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:22.513 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:22.521 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:22.565 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:22.564+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:22.665 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:22.664+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:22.749 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:22.749 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:22.750 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 369.41059120199998, 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.751 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.752 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.753 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.754 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:22.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:22.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:22.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:22.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:22.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:22.766 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.769 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 367.08943601099998, 2023-10-30T06:30:22.770 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.771 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:22.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:22.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:22.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:22.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:22.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:22.783 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:22.784 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.785 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 367.08519145399998, 2023-10-30T06:30:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.786 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:22.787 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:22.788 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:22.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:22.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:22.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:22.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:22.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:22.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.99973302900003, 2023-10-30T06:30:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.803 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:22.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.816 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.73048981300002, 2023-10-30T06:30:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:22.817 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:22.819 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:22.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:22.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.72965840199998, 2023-10-30T06:30:22.832 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.834 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:22.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:22.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:22.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:22.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:22.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:22.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:22.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.847 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.728991275, 2023-10-30T06:30:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.848 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.849 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:22.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:22.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.861 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:22.862 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.70671720399997, 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.863 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:22.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:22.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:22.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:22.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.70606934800003, 2023-10-30T06:30:22.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.878 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.879 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.880 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:22.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:22.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:22.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.891 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 366.70538327499997, 2023-10-30T06:30:22.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.893 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:22.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:22.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:22.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:22.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:22.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:22.905 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.906 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 365.63450130699999, 2023-10-30T06:30:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.907 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.908 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:22.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:22.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:22.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 365.633837341, 2023-10-30T06:30:22.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.922 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:22.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:22.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:22.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:22.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:22.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:22.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:22.934 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.935 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 365.60538353200002, 2023-10-30T06:30:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:22.936 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.937 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:22.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:22.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:22.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:22.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:22.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:22.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:22.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:22.946 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:22.947 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:22.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:22.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:22.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.263617039, 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.952 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:22.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:22.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:22.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:22.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:22.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:22.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:22.959 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:22.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:22.961 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:22.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:22.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.966 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:22.967 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.26286836899999, 2023-10-30T06:30:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.968 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.969 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.976 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.977 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.980 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.981 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.26219998599998, 2023-10-30T06:30:22.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.984 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:22.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:22.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:22.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:22.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:22.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:22.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.992 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:22.993 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:22.994 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:22.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:22.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:22.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.25234931199998, 2023-10-30T06:30:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:22.998 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:22.999 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:23.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:23.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:23.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:23.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:23.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:23.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:23.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:23.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:23.008 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:23.009 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:23.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.012 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:23.013 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.25167549700001, 2023-10-30T06:30:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:23.014 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:23.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:23.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:23.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:23.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:23.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.022 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.024 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:23.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:23.028 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.25102711400001, 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:23.029 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:23.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:23.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:23.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:23.038 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:23.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:23.040 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:23.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:23.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.043 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 358.24691064799998, 2023-10-30T06:30:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:23.044 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.045 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:23.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:23.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:23.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:23.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:23.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:23.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:23.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:23.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:23.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:23.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:23.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:23.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.053 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:23.053 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:23.053 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:23.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.054 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.054 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:23.054 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:23.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:23.054 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.055 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.055 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:23.055 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:23.055 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:23.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:23.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:23.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:23.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:23.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:23.058 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.059 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:23.059 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:23.059 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:23.061 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:23.060+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:23.161 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:23.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:23.228 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:23.228 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:23.229 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:23.229 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:23.262 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:23.262+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:23.363 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:23.363+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:23.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2023-10-30T06:30:23.422 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:23.422 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:23.423 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:23.465 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:23.464+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:23.543 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:30:23.565 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:23.564+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:23.612 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:23.612 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:23.613 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:23.706 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:23.704+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:23.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:23.790 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:23.788+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:23.792 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:23.793 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:23.793 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:23.867 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:23.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:23.969 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:23.968+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:23.976 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:23.976 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:23.976 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:23.976 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:23.976 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:23.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:24.032 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:24.032 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.069 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:24.068+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:24.147 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:24.147 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.148 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 287.85298057300002, 2023-10-30T06:30:24.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:24.150 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:24.151 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.152 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:24.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:24.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:24.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:24.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:24.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:24.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.162 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:24.163 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:24.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:24.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:24.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 107.820806354, 2023-10-30T06:30:24.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:24.169 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.175 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.176 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:24.177 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:24.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:24.179 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:24.179 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:24.180 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:24.182 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:24.180+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:24.253 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:24.253 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.281 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:24.281+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:24.312 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:24.313 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:24.313 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:24.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:24.337 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:24.338 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.382 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:24.381+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:24.427 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:24.427 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2023-10-30T06:30:24.514 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:24.515 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.515 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:24.514+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:24.522 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:24.523 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:24.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:24.600 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:24.601 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.617 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:24.616+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:24.722 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:24.722 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:24.724 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:24.724 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:24.726 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:24.725+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:24.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:24.785 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:24.783+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:24.886 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:24.884+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:24.897 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:24.898 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:24.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:24.987 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:24.986+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:25.042 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.042 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:25.089 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:25.087+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:25.176 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.177 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:25.189 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:25.188+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:25.290 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:25.289+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:25.311 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.311 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:25.391 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:25.391+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:25.448 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.449 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:25.499 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:25.499+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:25.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2023-10-30T06:30:25.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:30:25.594 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:25.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:25.627 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.627 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:25.726 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:25.725+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:25.796 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:25.794+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:25.809 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.810 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:25.897 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:25.895+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:25.985 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:25.986 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:25.995 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:25.998 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:25.997+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:26.098 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:26.097+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:26.124 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.124 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:26.199 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:26.198+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:26.265 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.265 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:26.319 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:26.317+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:26.401 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:26.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:26.406 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.406 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:26.502 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:26.501+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:26.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2023-10-30T06:30:26.592 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.592 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:26.603 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:26.601+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:26.618 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:30:26.703 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:26.702+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:26.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:26.749 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.749 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.759 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:26.846 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:26.844+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:26.905 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:26.903+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:26.943 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:26.944 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:26.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:26.993 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:26.994 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.038 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:27.036+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:27.101 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:27.101 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.107 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:27.105+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:27.204 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:27.204 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.208 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:27.207+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:27.229 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:27.229 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:27.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 373.88123900400001, 2023-10-30T06:30:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:27.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:27.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:27.246 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.247 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.56008381300001, 2023-10-30T06:30:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:27.248 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:27.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:27.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:27.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:27.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:27.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:27.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:27.255 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:27.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:27.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:27.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.55583925600001, 2023-10-30T06:30:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:27.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:27.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:27.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.275 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.278 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:27.279 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.280 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:27.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.284 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.470380831, 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.285 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.286 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.287 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.288 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.290 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.291 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.292 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.293 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.295 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.297 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.20113761499999, 2023-10-30T06:30:27.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.301 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.302 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:27.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:27.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:27.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:27.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:27.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.312 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.313 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:27.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.20030620400001, 2023-10-30T06:30:27.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.321 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.322 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:27.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:27.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:27.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:27.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:27.334 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.19963907699997, 2023-10-30T06:30:27.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.337 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.338 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:27.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:27.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:27.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:27.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:27.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:27.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:27.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.351 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:27.352 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.177365006, 2023-10-30T06:30:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.353 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:27.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:27.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:27.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:27.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:27.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:27.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:27.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:27.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:27.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:27.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:27.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.375 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.376 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.17671715, 2023-10-30T06:30:27.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.378 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:27.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:27.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:27.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.393 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:27.394 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 371.176031077, 2023-10-30T06:30:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.395 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:27.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:27.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.408 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:27.409 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:27.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 370.10514910900002, 2023-10-30T06:30:27.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.415 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:27.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:27.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:27.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.423 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.424 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.425 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:27.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 370.10448514299998, 2023-10-30T06:30:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:27.429 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.430 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:27.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:27.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:27.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:27.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:27.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:27.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:27.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.447 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 370.07603133399999, 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.448 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.449 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:27.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.464 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.73426484100003, 2023-10-30T06:30:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:27.465 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.466 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:27.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.471 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:27.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:27.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:27.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:27.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:27.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.490 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:27.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.73351617100002, 2023-10-30T06:30:27.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.496 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:27.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.510 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:27.511 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.73284778800002, 2023-10-30T06:30:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.512 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.521 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.522 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.523 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.72299711400001, 2023-10-30T06:30:27.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.529 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.530 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:27.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:27.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:27.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:27.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:27.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:27.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.546 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.72232329899998, 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:27.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:27.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:27.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:27.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:27.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:27.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:27.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:27.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.72167491599998, 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.563 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:27.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:27.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:27.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:27.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:27.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:27.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.577 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:27.578 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 362.71755845000001, 2023-10-30T06:30:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:27.579 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:27.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.586 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.587 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:27.588 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:27.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:27.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:27.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:27.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:27.593 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:27.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:27.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:27.595 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2023-10-30T06:30:27.595 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-10-30T06:30:27.595 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-10-30T06:30:27.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2023-10-30T06:30:27.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:27.598 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:27.598 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.599 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:27.599 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.600 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:27.600 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.600 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:27.601 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.601 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:27.593+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:27.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2023-10-30T06:30:27.630 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:27.630 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:27.697 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:27.695+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:27.764 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:27.765 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:27.765 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:27.769 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:27.769 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "" 2023-10-30T06:30:27.770 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:27.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:27.797 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:27.796+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:27.899 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:27.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:27.945 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:27.946 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:27.947 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:27.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:28.039 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:28.037+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:28.100 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:28.098+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:28.142 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:28.143 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:28.143 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:28.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:28.237 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:28.235+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:28.301 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:28.299+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:28.335 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:28.336 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:28.336 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:28.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:28.401 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:28.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:28.503 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:28.502+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:28.511 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:28.512 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:28.512 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:28.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:28.607 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:28.606+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:28.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2023-10-30T06:30:28.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:30:28.697 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:28.697 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:28.698 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 292.40354047599999, 2023-10-30T06:30:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:28.699 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:28.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.706 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:28.711 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:28.712 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 112.37136625700001, 2023-10-30T06:30:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:28.713 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:28.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.715 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.718 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:28.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:28.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:28.722 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:28.723 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:28.724 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:28.724 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:28.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:28.726 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:28.724+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:28.827 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:28.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:28.861 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:28.861 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:28.861 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:28.862 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:28.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:28.928 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:28.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:29.029 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:29.028+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:29.039 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:29.039 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:29.039 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:29.040 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:29.040 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:29.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:29.132 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:29.131+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:29.231 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:29.231 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:29.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:29.231 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:29.232 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:29.232 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:29.230+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:29.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:29.332 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:29.330+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:29.412 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:29.412 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:29.412 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:29.413 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:29.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:29.433 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:29.432+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:29.534 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:29.533+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:29.550 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:29.550 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:29.551 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:29.551 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:29.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:29.635 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:29.633+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:29.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2023-10-30T06:30:29.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:29.737 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:29.735+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:29.767 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:29.768 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:29.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:29.768 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:29.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.769 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:29.769 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:29.769 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.61060911099997, 2023-10-30T06:30:29.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:29.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:29.770 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:29.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:29.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.776 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:29.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:29.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:29.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:29.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:29.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.788 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:29.789 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 374.69743524799998, 2023-10-30T06:30:29.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.792 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.799 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.800 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.802 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.803 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:29.803 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:29.803 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 374.03725727900002, 2023-10-30T06:30:29.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:29.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.804 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:29.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.806 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:29.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:29.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:29.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:29.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:29.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:29.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:29.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.812 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:29.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:29.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.812 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.813 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:29.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:29.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.813 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:29.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:29.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.814 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.814 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.815 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:29.815 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:29.815 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 373.96183407199999, 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:29.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:29.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:29.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:29.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:29.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:29.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:29.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:29.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:29.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.824 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.825 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:29.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:29.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:29.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:29.826 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.826 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.826 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.827 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:29.827 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:29.827 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 373.69071094100002, 2023-10-30T06:30:29.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:29.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.828 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:29.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:29.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:29.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:29.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:29.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:29.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:29.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:29.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:29.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:29.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:29.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:29.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.838 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.838 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 373.66043123100002, 2023-10-30T06:30:29.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:29.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:29.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:29.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:29.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:29.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:29.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.845 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:29.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:29.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:29.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:29.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:29.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:29.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:29.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:29.850 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.851 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.851 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.851 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:29.852 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:29.852 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 373.28652014699998, 2023-10-30T06:30:29.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:29.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.853 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:29.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:29.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:29.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:29.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:29.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:29.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:29.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:29.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:29.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:29.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:29.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:29.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:29.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:29.862 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.863 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.863 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:29.864 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:29.864 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 373.28202144599999, 2023-10-30T06:30:29.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:29.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:29.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:29.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:29.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:29.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:29.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:29.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:29.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:29.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:29.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:29.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:29.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:29.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:29.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:29.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:29.874 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.874 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.875 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:29.875 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:29.876 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.99041750700002, 2023-10-30T06:30:29.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:29.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.876 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:29.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:29.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:29.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:29.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:29.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:29.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:29.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:29.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:29.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:29.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:29.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.886 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.887 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:29.887 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:29.887 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.98448019, 2023-10-30T06:30:29.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:29.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.888 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:29.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:29.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:29.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:29.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:29.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:29.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:29.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.893 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:29.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:29.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:29.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:29.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:29.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:29.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:29.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.898 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:29.899 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:29.899 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.98318117500003, 2023-10-30T06:30:29.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:29.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.900 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:29.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:29.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:29.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:29.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:29.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:29.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:29.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:29.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:29.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:29.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:29.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.910 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:29.910 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:29.910 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.88019236700001, 2023-10-30T06:30:29.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:29.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:29.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:29.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:29.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:29.920 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.921 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.60828909999998, 2023-10-30T06:30:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:29.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:29.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:29.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:29.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:29.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:29.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:29.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:29.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:29.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.932 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.932 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.933 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:29.933 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:29.933 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.60079090800002, 2023-10-30T06:30:29.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:29.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.934 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:29.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.939 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.939 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:29.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.940 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.940 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:29.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:29.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.941 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:29.941 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:29.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:29.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:29.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:29.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:29.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.946 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:29.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:29.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:29.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:29.947 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.947 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.948 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.948 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:29.948 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.60012970299999, 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:29.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.951 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.952 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:29.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:29.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.952 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:29.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:29.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:29.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:29.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:29.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:29.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.959 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.959 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.58754386499999, 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.961 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:29.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:29.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.963 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.963 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:29.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:29.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.965 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.965 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:29.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:29.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:29.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:29.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:29.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:29.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:29.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:29.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.970 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.970 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:29.971 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.581451652, 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:29.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:29.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:29.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.976 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.976 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:29.977 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:29.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.977 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:29.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:29.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:29.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:29.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:29.981 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.982 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.982 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.982 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:29.983 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:29.983 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 372.58045631099998, 2023-10-30T06:30:29.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:29.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.984 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:29.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:29.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:29.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.987 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.987 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.987 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:29.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:29.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:29.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:29.989 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:29.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:29.990 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:29.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:29.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:29.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:29.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:29.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:29.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:29.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:29.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:29.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:29.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.994 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:29.994 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:29.994 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 365.25778373999998, 2023-10-30T06:30:29.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:29.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:29.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:29.995 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:29.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:29.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:29.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:29.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:29.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:29.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:29.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:29.999 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:30.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:30.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.000 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.000 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:30.000 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:30.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:30.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:30.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:30.002 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:30.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:30.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:30.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:30.004 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:30.005 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:30.005 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:30.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.005 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:30.006 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:30.006 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 365.22434225500001, 2023-10-30T06:30:30.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:30.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:30.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:30.007 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:30.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:30.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:30.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:30.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:30.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.011 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:30.011 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:30.011 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:30.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.012 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:30.012 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:30.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:30.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.012 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:30.125 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:30.126 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:30.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.126 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.127 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:30.127 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:30.127 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:30.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.128 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:30.128 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:30.129 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:30.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:30.129 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:30.130 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:30.130 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:30.131 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:30.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:30.131 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:30.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:30.132 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:30.132 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:30.133 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:30.134 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:30.135 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.136 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:30.137 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:30.138 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:30.138 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.142 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:30.133+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:30.235 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:30.234+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:30.242 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:30.242 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.245 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:30:30.324 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:30.324 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:30.325 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:30.325 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:30.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:30.350 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:30.350 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.376 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:30.375+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:30.439 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:30.438+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:30.450 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:30.451 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.540 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:30.540 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.541 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:30.541 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:30.541 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:30.539+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:30.551 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:30.630 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:30.631 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:30.641 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:30.640+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:30.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2023-10-30T06:30:30.718 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:30.718 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:30.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:30.743 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:30.741+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:30.843 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:30.842+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:30.900 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:30.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:30.953 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:30.950+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:31.045 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:31.044+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:31.077 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.077 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.087 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:31.146 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:31.145+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:31.215 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.215 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:31.269 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:31.267+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:31.348 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:31.346+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:31.367 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.368 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:31.460 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:31.459+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:31.542 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.543 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.549 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:31.548+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:31.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:31.650 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:31.649+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:31.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2023-10-30T06:30:31.726 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.727 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:31.751 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:31.750+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:31.852 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:31.851+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:31.905 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:31.905 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:31.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:31.956 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:31.954+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:32.054 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:32.053+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:32.083 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.083 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:32.155 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:32.154+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:32.256 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:32.255+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:32.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:30:32.260 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.261 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:32.357 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:32.356+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:32.444 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.445 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:32.458 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:32.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:32.560 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:32.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:32.620 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.620 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:32.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:32.661 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:32.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:32.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2023-10-30T06:30:32.762 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:32.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:32.767 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.767 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:32.780 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:32.780 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:32.865 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:32.863+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:32.947 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:32.948 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:32.964 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:32.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:32.967 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:32.968 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:32.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:33.036 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:33.036 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.065 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:33.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:33.150 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:33.150 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.166 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:33.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:33.179 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:33.180 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:33.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:33.255 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:33.255 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.266 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:33.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:33.338 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:33.339 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.365 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:33.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 297.07211291300001, 2023-10-30T06:30:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:33.367 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:33.368 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:33.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:33.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:33.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:33.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:33.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:33.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:33.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:33.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:33.378 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:33.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 117.039938694, 2023-10-30T06:30:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:33.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:33.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:33.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:33.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:33.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:33.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:33.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:33.391 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:33.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:33.394 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:33.395 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:33.395 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:33.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:33.397 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:33.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:33.423 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.498 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:33.497+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:33.509 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:33.510 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.526 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:33.527 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:33.527 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:33.528 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:33.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:33.594 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:33.595 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:33.598 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:33.597+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:33.667 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:33.668 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:33.668 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:33.669 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:33.669 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:33.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:33.699 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:33.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:33.800 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:33.798+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:33.808 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:33.809 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:33.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:33.809 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:33.809 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:33.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:33.841 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:30:33.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2023-10-30T06:30:33.901 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:33.899+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:33.946 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:33.946 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:33.947 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:33.947 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:33.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:34.041 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:34.039+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:34.103 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:34.102+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:34.127 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:34.127 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:34.128 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:34.128 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:34.137 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:34.204 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:34.203+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:34.304 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:34.303+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:34.348 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:34.348 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:34.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:34.348 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:34.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.349 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:34.349 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:34.349 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 382.19189799399999, 2023-10-30T06:30:34.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:34.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:34.350 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:34.350 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:34.351 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:34.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:34.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.351 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:34.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:34.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:34.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:34.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:34.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.358 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.358 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:34.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:34.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.359 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:34.359 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:34.359 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:34.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.360 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.360 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:34.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:34.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:34.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:34.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.362 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.362 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.363 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:34.364 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:34.364 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 379.27872413099999, 2023-10-30T06:30:34.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:34.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.366 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:34.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:34.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.370 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.370 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.371 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:34.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:34.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:34.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:34.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:34.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:34.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:34.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:34.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:34.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:34.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:34.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:34.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.382 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.382 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:34.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:34.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:34.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:34.385 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.386 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.386 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.387 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:34.388 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:34.388 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 378.61854616199997, 2023-10-30T06:30:34.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:34.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.390 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:34.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:34.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:34.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:34.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.394 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.394 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:34.394 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:34.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:34.396 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:34.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:34.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:34.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:34.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:34.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:34.400 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.401 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.401 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.402 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:34.402 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:34.402 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 378.543122955, 2023-10-30T06:30:34.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:34.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:34.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:34.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.406 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:34.406 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:34.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.406 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:34.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:34.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:34.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:34.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:34.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:34.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:34.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:34.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:34.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:34.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.413 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.414 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:34.414 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:34.414 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 378.27199982399998, 2023-10-30T06:30:34.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:34.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.415 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:34.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:34.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.417 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.418 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.418 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:34.418 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:34.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.419 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.419 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:34.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:34.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:34.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:34.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:34.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:34.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:34.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:34.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:34.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:34.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:34.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:34.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.425 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.426 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:34.426 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:34.426 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 378.24172011399997, 2023-10-30T06:30:34.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:34.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.427 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:34.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.429 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.429 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.429 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:34.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.430 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.430 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:34.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:34.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:34.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:34.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:34.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:34.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:34.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:34.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.435 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:34.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:34.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:34.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:34.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.86780902999999, 2023-10-30T06:30:34.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.442 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.443 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:34.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:34.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:34.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:34.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:34.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:34.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:34.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:34.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:34.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:34.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.451 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.452 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.452 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:34.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:34.453 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.453 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.454 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.455 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:34.455 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:34.456 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.863310329, 2023-10-30T06:30:34.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:34.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.457 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:34.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:34.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.462 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.462 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.462 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:34.463 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:34.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.464 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.464 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.464 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:34.465 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:34.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.466 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.466 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.466 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:34.467 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:34.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.467 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.468 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.468 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:34.469 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:34.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.469 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.470 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.470 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:34.470 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:34.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.472 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.472 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:34.472 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:34.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.473 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.474 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:34.474 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:34.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.475 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.476 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:34.476 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:34.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.477 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.477 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.478 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:34.478 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:34.479 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.57170638999997, 2023-10-30T06:30:34.479 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:34.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.479 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.481 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.481 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:34.481 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.482 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.482 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.482 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:34.482 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:34.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.483 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.483 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.483 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:34.483 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:34.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.484 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.484 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.484 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:34.484 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:34.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.485 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.485 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.485 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:34.485 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:34.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.487 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.487 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.487 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:34.487 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:34.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.488 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.488 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.488 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:34.488 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:34.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.489 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.489 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.489 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:34.489 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:34.489 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.490 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:34.491 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:34.491 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.56576907300001, 2023-10-30T06:30:34.491 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:34.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.493 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.493 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:34.493 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.494 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.494 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.494 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:34.494 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:34.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.495 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.495 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.495 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:34.495 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:34.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.496 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.496 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.496 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:34.496 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:34.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.497 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.497 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.497 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:34.497 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:34.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.498 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.498 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.498 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:34.498 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:34.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.499 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.499 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:34.499 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:34.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.500 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.500 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.500 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:34.500 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:34.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.502 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.502 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.502 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.503 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:34.503 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:34.503 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.56447005799998, 2023-10-30T06:30:34.503 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:34.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.504 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.504 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.504 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:34.505 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.505 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.505 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.505 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:34.506 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.506 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.506 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:34.507 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:34.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.507 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.507 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.507 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:34.508 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:34.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.508 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.508 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:34.509 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:34.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.510 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.510 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.510 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:34.511 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:34.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.511 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.511 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.511 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:34.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.513 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.513 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.513 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:34.513 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:34.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.514 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.515 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.515 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:34.515 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:34.515 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.46148125000002, 2023-10-30T06:30:34.515 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:34.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.516 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.516 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.517 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:34.517 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.517 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.518 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:34.518 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.518 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.518 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.519 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:34.519 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:34.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.519 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.520 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:34.520 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:34.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.520 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.520 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.521 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:34.521 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:34.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.521 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.521 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.522 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:34.522 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:34.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.522 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.523 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.523 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:34.523 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:34.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.523 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.524 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.525 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:34.525 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:34.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.526 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.526 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.526 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:34.527 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:34.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.527 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.527 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.528 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.528 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:34.528 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:34.528 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.18957798299999, 2023-10-30T06:30:34.528 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:34.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.529 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.529 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.529 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.530 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:34.530 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.530 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.531 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:34.532 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:34.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.532 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.532 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.532 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:34.533 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:34.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.534 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.534 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.534 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:34.534 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:34.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.535 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.535 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.535 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:34.535 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:34.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:34.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.538 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.538 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.538 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:34.538 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:34.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.539 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.539 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.18207979099998, 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.540 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.541 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.541 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.541 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:34.541 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.542 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.542 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:34.542 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.543 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.543 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:34.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:34.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.544 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.544 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.544 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:34.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:34.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.546 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.546 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:34.546 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:34.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.547 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.547 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:34.547 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:34.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:34.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:34.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:34.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:34.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:34.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:34.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.551 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.552 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:34.552 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:34.552 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.18141858600001, 2023-10-30T06:30:34.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:34.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.553 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:34.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:34.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:34.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:34.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.558 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.558 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:34.558 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:34.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.559 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.559 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:34.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:34.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.559 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:34.560 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:34.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:34.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:34.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:34.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:34.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:34.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:34.563 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.564 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.564 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.564 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.168832748, 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.565 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:34.566 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:34.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:34.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:34.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:34.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:34.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:34.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:34.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:34.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:34.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:34.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:34.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:34.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.576 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.576 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.577 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:34.577 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:34.577 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.16274053500001, 2023-10-30T06:30:34.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:34.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.578 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:34.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:34.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:34.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:34.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:34.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:34.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.583 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.583 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:34.583 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:34.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.584 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.584 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:34.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:34.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:34.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:34.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:34.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:34.587 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.588 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.588 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:34.589 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:34.589 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 377.16174519399999, 2023-10-30T06:30:34.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:34.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.590 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:34.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:34.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:34.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:34.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.594 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:34.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:34.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.596 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.596 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:34.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:34.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:34.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:34.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:34.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:34.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:34.599 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:34.600 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:34.601 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 369.83907262299999, 2023-10-30T06:30:34.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:34.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.601 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:34.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:34.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:34.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:34.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:34.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:34.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.606 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:34.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:34.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.607 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.607 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:34.607 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:34.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.608 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.608 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:34.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:34.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:34.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:34.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:34.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:34.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:34.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.612 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.613 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:34.613 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:34.613 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 369.80563113800002, 2023-10-30T06:30:34.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:34.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:34.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:34.614 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:34.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:34.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:34.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:34.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:34.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:34.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:34.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:34.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:34.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:34.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:34.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:34.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:34.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:34.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:34.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:34.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:34.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:34.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:34.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:34.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:34.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:34.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:34.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:34.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:34.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:34.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.624 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:34.625 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:34.625 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:34.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:34.627 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:34.625+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:34.728 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:34.726+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:34.797 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:34.797 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:34.798 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:34.798 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:34.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:34.828 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:34.827+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:34.858 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2023-10-30T06:30:34.929 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:34.928+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:34.934 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:34.934 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:34.935 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:34.935 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:34.935 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:34.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:35.030 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:35.029+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:35.120 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:35.120 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:35.120 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:35.121 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:35.121 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:35.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:35.170 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:35.168+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:35.231 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:35.230+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:35.297 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:35.297 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:35.298 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:35.298 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:35.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:35.350 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:35.348+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:35.432 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:35.433 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:35.433 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:35.433 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:35.433 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:35.434 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:35.432+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:35.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:35.534 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:35.534+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:35.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:35.611 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:35.611 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:35.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:35.612 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:35.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.612 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:35.612 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:35.612 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 298.63740344500002, 2023-10-30T06:30:35.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:35.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:35.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:35.613 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:35.613 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:35.614 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:35.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:35.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:35.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:35.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:35.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:35.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:35.617 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:35.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:35.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.618 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.618 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:35.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:35.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:35.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.619 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:35.619 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:35.619 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:35.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:35.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:35.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:35.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.622 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:35.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:35.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:35.622 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:35.622 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:35.623 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 118.616911121, 2023-10-30T06:30:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:35.624 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:35.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:35.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:35.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:35.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:35.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:35.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:35.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:35.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:35.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:35.634 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:35.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:35.636 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:35.634+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:35.736 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:35.736+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:35.796 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:35.796 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:35.796 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:35.796 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:35.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:35.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2023-10-30T06:30:35.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:30:35.858 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:35.858 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:35.859 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:35.857+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:35.938 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:35.936+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:35.974 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:35.974 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.012 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.013 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:36.071 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:36.070+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:36.086 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:36.086 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.140 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:36.139+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:36.157 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.158 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:36.192 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:36.193 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.241 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:36.240+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:36.294 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:36.295 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.342 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:36.342+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:36.353 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.353 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:36.389 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:36.389 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.443 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:36.442+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:36.490 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:36.490 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.533 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.533 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:36.544 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:36.542+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:36.591 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:36.592 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:36.644 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:36.644+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:36.674 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.674 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.684 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:36.767 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:36.767+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:36.846 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:36.845+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:36.855 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:36.856 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:36.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:36.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2023-10-30T06:30:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:30:36.948 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:36.946+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.017 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:37.049 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:37.048+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:37.150 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:37.149+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:37.155 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.155 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:37.251 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:37.250+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:37.352 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:37.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.360 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:37.453 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:37.452+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:37.536 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.536 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:37.591 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:37.590+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:37.655 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:37.654+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:37.675 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.675 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:37.756 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:37.755+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:37.781 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:37.782 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:37.782 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2023-10-30T06:30:37.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:30:37.814 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.815 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:37.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2023-10-30T06:30:37.868 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:37.866+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:37.952 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:37.953 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:37.957 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:37.957+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:37.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:38.059 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:38.057+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:38.133 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:38.134 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:38.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:38.160 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:38.159+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:38.263 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:38.262+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:38.316 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:38.316 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:38.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:38.362 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:38.361+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:38.419 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:30:38.420 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 113, 2023-10-30T06:30:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:30:38.420 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:30:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:30:36.862379+0000", 2023-10-30T06:30:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:30:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:30:13.065550+0000", 2023-10-30T06:30:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:30:38.421 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5806080, 2023-10-30T06:30:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:30:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2023-10-30T06:30:38.422 INFO:teuthology.orchestra.run.smithi079.stdout: "noscrub", 2023-10-30T06:30:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:30:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:30:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:30:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:30:38.423 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:30:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:30:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:30:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:30:38.424 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:30:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:30:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:30:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:30:38.425 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:30:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:30:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:30:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:30:38.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:30:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:30:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:30:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:30:38.427 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:30:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:30:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:30:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:30:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:30:38.428 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:30:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:30:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:30:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:30:38.429 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:30:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:30:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:30:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:30:38.430 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:30:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:30:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:30:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:30:38.431 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:30:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:30:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:30:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:30:38.432 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:30:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:30:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:30:38.433 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:30:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:30:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:30:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:30:38.434 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:30:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:30:38.435 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:30:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:30:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:30:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:30:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:30:38.436 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:30:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:30:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:30:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:30:38.437 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:30:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:30:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:30:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:30:38.438 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:30:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:30:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:30:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:30:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:30:38.439 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:30:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:30:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:30:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:30:38.440 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:30:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:30:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:30:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:30:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:30:38.441 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:30:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:30:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:30:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:30:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:30:38.442 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:30:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:30:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:30:38.443 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:30:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:30:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:30:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:30:38.444 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:30:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:30:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:30:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.77820587158203125, 2023-10-30T06:30:38.445 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.88999998569488525, 2023-10-30T06:30:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.77999997138977051 2023-10-30T06:30:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.446 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:30:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:30:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:30:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:30:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:30:38.447 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:30:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:30:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:30:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:30:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:30:38.448 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:30:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:30:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:30:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:30:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:30:38.449 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:30:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:30:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:30:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:30:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:30:38.450 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:30:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:30:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:30:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:30:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:30:38.451 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:30:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:30:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:30:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:30:38.452 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:30:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:30:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:30:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:30:38.453 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:30:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:30:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:30:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:30:38.454 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:30:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:30:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:30:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:30:38.455 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:30:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:30:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:30:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:30:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:30:38.456 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:30:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:30:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:30:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:30:38.457 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:30:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:30:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:30:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:30:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.458 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:30:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:30:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:30:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:30:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:30:38.459 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:30:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:30:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:30:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:30:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:30:38.460 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:30:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:30:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:30:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:30:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.461 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:30:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 3.5099999904632568, 2023-10-30T06:30:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 3.5099999904632568, 2023-10-30T06:30:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 1.1399999856948853, 2023-10-30T06:30:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 4, 2023-10-30T06:30:38.462 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 4, 2023-10-30T06:30:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.77820587158203125, 2023-10-30T06:30:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.88999998569488525, 2023-10-30T06:30:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.77999997138977051 2023-10-30T06:30:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.463 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:30:38.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:30:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:30:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.465 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.466 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.467 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:30:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.470 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:30:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.471 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.472 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.473 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:30:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.475 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:30:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.476 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:30:38.477 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:30:38.480 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:30:38.481 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:30:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:30:38.482 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:30:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:30:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:30:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.483 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:30:38.484 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:30:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.485 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:30:38.486 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:30:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:30:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.489 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:30:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:30:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:30:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:30:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:30:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:30:38.498 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:30:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:30:38.499 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:30:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:30:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:30:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.500 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.501 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:30:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:30:38.502 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.503 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:30:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.504 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.505 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:30:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:30:38.508 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.509 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:30:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.512 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:30:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.513 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.514 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.515 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:30:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.516 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:30:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.518 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:30:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.519 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:30:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:30:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.520 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.521 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.522 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:30:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:30:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.523 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:38.525 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 104, 2023-10-30T06:30:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.526 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:30:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.528 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:30:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:30:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.531 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.532 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:30:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.533 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:30:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.535 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:30:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.536 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:30:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:30:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:30:38.538 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:30:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:30:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:30:38.539 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.540 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.541 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:30:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:30:38.542 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.543 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.544 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:30:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.545 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.547 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:30:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:30:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.549 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:30:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:30:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.552 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:30:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:30:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:30:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:30:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:30:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:30:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:30:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:30:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:30:38.559 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.561 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:38.562 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 69, 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.563 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:30:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:30:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.565 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.566 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:30:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.568 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:30:38.569 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:30:38.570 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:30:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:30:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:30:38.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:30:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:30:38.574 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:30:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:30:38.575 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:30:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:30:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.577 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.22564697265625, 2023-10-30T06:30:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.578 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:30:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:30:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.579 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.580 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:30:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.581 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:30:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.582 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:30:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.584 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:30:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.585 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:30:38.586 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:30:38.587 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:30:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:30:38.590 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:30:38.591 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:30:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:30:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.592 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:30:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:30:38.593 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:30:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:30:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:30:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:30:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:30:38.594 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:30:38.595 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:30:38.595 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 37, 2023-10-30T06:30:38.595 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:30:38.595 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:30:38.595 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:30:38.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.596 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.596 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:30:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.597 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:30:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.598 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:30:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:30:38.599 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:30:38.600 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:30:38.601 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:30:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:30:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:30:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:30:38.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:30:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:30:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:30:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:30:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:30:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.606 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:30:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:30:38.607 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:30:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:30:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:30:38.608 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:30:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:30:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.609 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:30:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.611 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:30:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:30:38.612 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.613 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:30:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.614 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:30:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.615 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:30:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.616 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:30:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.617 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:30:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:30:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.619 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:30:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.621 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:30:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.623 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:30:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.624 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:30:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:30:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:30:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:30:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:30:38.625 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:30:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:30:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:30:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.626 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:30:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:30:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:30:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: 1, 2023-10-30T06:30:38.627 INFO:teuthology.orchestra.run.smithi079.stdout: 2 2023-10-30T06:30:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:30:38.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 5, 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2023-10-30T06:30:38.629 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:30:38.630 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:30:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:30:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:30:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:30:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:30:38.631 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:30:38.632 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:30:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:30:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:30:38.633 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:30:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:30:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:30:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:30:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:30:38.634 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:30:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:30:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:30:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:30:38.635 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:38.635 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:38.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:38.638 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:38.638 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:38.639 INFO:tasks.thrashosds.thrasher:j is {'epoch': 113, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:30:36.862379+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:30:13.065550+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7782058715820312, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.7799999713897705}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.509999990463257, 'score_stable': 3.509999990463257, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 4, 'raw_score_stable': 4, 'primary_affinity_weighted': 0.7782058715820312, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.7799999713897705}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.22564697265625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [1, 2]}], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:30:38.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:30:38.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:38.641 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:38.636+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:38.738 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:38.738+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:38.809 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:38.809 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:38.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:38.863 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:38.861+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:38.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2023-10-30T06:30:38.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:30:38.939 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:38.940 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:38.940 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:38.938+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:38.949 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:38.950 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:38.959 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:39.043 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:39.042+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:39.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:39.045 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.128 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:39.128 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:39.134 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:39.135 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:39.179 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:39.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:39.224 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:39.225 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.243 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:39.242+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:39.285 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:30:39.286 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:30:39.299 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-10-30T06:30:37.321378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221948,"kb_used_data":5788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752772,"statfs":{"total":773094113280,"available":772866838528,"internally_reserved":0,"allocated":5926912,"data_stored":3420191,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.074612"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-30T06:30:30.375930+0000","last_change":"2023-10-30T06:30:14.747570+0000","last_active":"2023-10-30T06:30:30.375930+0000","last_peered":"2023-10-30T06:30:30.375930+0000","last_clean":"2023-10-30T06:30:30.375930+0000","last_became_active":"2023-10-30T06:30:14.741239+0000","last_became_peered":"2023-10-30T06:30:14.741239+0000","last_unstale":"2023-10-30T06:30:30.375930+0000","last_undegraded":"2023-10-30T06:30:30.375930+0000","last_fullsized":"2023-10-30T06:30:30.375930+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:58:53.403617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":99,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-30T06:30:30.279226+0000","last_change":"2023-10-30T06:24:20.726406+0000","last_active":"2023-10-30T06:30:30.279226+0000","last_peered":"2023-10-30T06:30:30.279226+0000","last_clean":"2023-10-30T06:30:30.279226+0000","last_became_active":"2023-10-30T06:24:20.689966+0000","last_became_peered":"2023-10-30T06:24:20.689966+0000","last_unstale":"2023-10-30T06:30:30.279226+0000","last_undegraded":"2023-10-30T06:30:30.279226+0000","last_fullsized":"2023-10-30T06:30:30.279226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:16:06.426977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-30T06:30:27.109666+0000","last_change":"2023-10-30T06:25:16.266355+0000","last_active":"2023-10-30T06:30:27.109666+0000","last_peered":"2023-10-30T06:30:27.109666+0000","last_clean":"2023-10-30T06:30:27.109666+0000","last_became_active":"2023-10-30T06:25:16.260730+0000","last_became_peered":"2023-10-30T06:25:16.260730+0000","last_unstale":"2023-10-30T06:30:27.109666+0000","last_undegraded":"2023-10-30T06:30:27.109666+0000","last_fullsized":"2023-10-30T06:30:27.109666+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:36:11.551486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":116,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-30T06:30:35.081645+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:30:35.081645+0000","last_peered":"2023-10-30T06:30:35.081645+0000","last_clean":"2023-10-30T06:30:35.081645+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:30:35.081645+0000","last_undegraded":"2023-10-30T06:30:35.081645+0000","last_fullsized":"2023-10-30T06:30:35.081645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-30T06:30:30.367074+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:30:30.367074+0000","last_peered":"2023-10-30T06:30:30.367074+0000","last_clean":"2023-10-30T06:30:30.367074+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:30:30.367074+0000","last_undegraded":"2023-10-30T06:30:30.367074+0000","last_fullsized":"2023-10-30T06:30:30.367074+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":100,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-30T06:30:34.199811+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:30:34.199811+0000","last_peered":"2023-10-30T06:30:34.199811+0000","last_clean":"2023-10-30T06:30:34.199811+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:30:34.199811+0000","last_undegraded":"2023-10-30T06:30:34.199811+0000","last_fullsized":"2023-10-30T06:30:34.199811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-30T06:30:30.369099+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:30:30.369099+0000","last_peered":"2023-10-30T06:30:30.369099+0000","last_clean":"2023-10-30T06:30:30.369099+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:30:30.369099+0000","last_undegraded":"2023-10-30T06:30:30.369099+0000","last_fullsized":"2023-10-30T06:30:30.369099+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-30T06:30:30.373070+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:30:30.373070+0000","last_peered":"2023-10-30T06:30:30.373070+0000","last_clean":"2023-10-30T06:30:30.373070+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:30:30.373070+0000","last_undegraded":"2023-10-30T06:30:30.373070+0000","last_fullsized":"2023-10-30T06:30:30.373070+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":204,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-30T06:30:27.105236+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:30:27.105236+0000","last_peered":"2023-10-30T06:30:27.105236+0000","last_clean":"2023-10-30T06:30:27.105236+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:30:27.105236+0000","last_undegraded":"2023-10-30T06:30:27.105236+0000","last_fullsized":"2023-10-30T06:30:27.105236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411408,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":721066,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":11.656000000000001,"5min":10.69,"15min":10.470000000000001},"min":{"1min":4.6680000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":41.893999999999998,"5min":51.225000000000001,"15min":51.225000000000001},"last":20.494},{"interface":"front","average":{"1min":9.5790000000000006,"5min":10.984,"15min":11.715999999999999},"min":{"1min":5.1260000000000003,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":19.98,"5min":54.011000000000003,"15min":54.011000000000003},"last":10.423}]},{"osd":2,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":11.449,"5min":11.566000000000001,"15min":13.054},"min":{"1min":6.8380000000000001,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":19.623999999999999,"5min":44.537999999999997,"15min":52.552},"last":12.414999999999999},{"interface":"front","average":{"1min":15.808,"5min":14.797000000000001,"15min":15.029999999999999},"min":{"1min":8.6440000000000001,"5min":7.391,"15min":7.1589999999999998},"max":{"1min":40.183,"5min":55.585999999999999,"15min":55.585999999999999},"last":18.687000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":22.440999999999999,"5min":16.683,"15min":21.021000000000001},"min":{"1min":10.663,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":48.716000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":15.502000000000001},{"interface":"front","average":{"1min":17.629000000000001,"5min":14.619,"15min":19.920999999999999},"min":{"1min":8.3030000000000008,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":50.332999999999998,"5min":50.332999999999998,"15min":124.075},"last":8.7720000000000002}]},{"osd":4,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":17.442,"5min":13.93,"15min":13.76},"min":{"1min":8.2159999999999993,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":48.134999999999998,"5min":48.134999999999998,"15min":51.649000000000001},"last":11.084},{"interface":"front","average":{"1min":18.248000000000001,"5min":17.190000000000001,"15min":17.728999999999999},"min":{"1min":9.1440000000000001,"5min":8.8810000000000002,"15min":7.9400000000000004},"max":{"1min":46.308,"5min":46.308,"15min":46.308},"last":7.6760000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":16.280999999999999,"5min":16.262,"15min":19.454999999999998},"min":{"1min":8.0500000000000007,"5min":7.351,"15min":7.351},"max":{"1min":26.207999999999998,"5min":58.648000000000003,"15min":123.754},"last":14.75},{"interface":"front","average":{"1min":17.007000000000001,"5min":15.577999999999999,"15min":20.579000000000001},"min":{"1min":6.8520000000000003,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":31.82,"5min":41.558999999999997,"15min":122.72499999999999},"last":16.446000000000002}]},{"osd":6,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":17.536999999999999,"15min":20.149999999999999},"min":{"1min":9.8979999999999997,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":27.460999999999999,"5min":39.860999999999997,"15min":66.400999999999996},"last":18.106999999999999},{"interface":"front","average":{"1min":19.936,"5min":17.399999999999999,"15min":19.039000000000001},"min":{"1min":11.220000000000001,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":32.485999999999997,"5min":43.159999999999997,"15min":53.149000000000001},"last":14.112}]},{"osd":7,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":22.295999999999999,"5min":19.056000000000001,"15min":20.693999999999999},"min":{"1min":11.384,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":43.174999999999997,"5min":51.375,"15min":51.375},"last":19.718},{"interface":"front","average":{"1min":19.138999999999999,"5min":19.327000000000002,"15min":20.838999999999999},"min":{"1min":8.6739999999999995,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":29.103999999999999,"5min":54.439,"15min":69.991},"last":11.792}]}]},{"osd":6,"up_from":16,"seq":68719476818,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":10.955,"5min":10.477,"15min":13.295999999999999},"min":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"max":{"1min":31.138000000000002,"5min":37.701999999999998,"15min":94.343000000000004},"last":8.157},{"interface":"front","average":{"1min":10.48,"5min":10.316000000000001,"15min":13.941000000000001},"min":{"1min":4.4029999999999996,"5min":4.4029999999999996,"15min":4.4029999999999996},"max":{"1min":22.327000000000002,"5min":49.045999999999999,"15min":122.054},"last":20.488}]},{"osd":1,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":11.297000000000001,"5min":10.913,"15min":10.848000000000001},"min":{"1min":5.1299999999999999,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":22.393000000000001,"5min":24.128,"15min":24.128},"last":7.3200000000000003},{"interface":"front","average":{"1min":13.782999999999999,"5min":13.946999999999999,"15min":27.225000000000001},"min":{"1min":8.0079999999999991,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":37.746000000000002,"5min":37.746000000000002,"15min":387.15600000000001},"last":10.816000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":14.441000000000001,"5min":13.300000000000001,"15min":17.736999999999998},"min":{"1min":8.6199999999999992,"5min":6.5800000000000001,"15min":6.4359999999999999},"max":{"1min":25.774999999999999,"5min":72.988,"15min":156.90299999999999},"last":11.601000000000001},{"interface":"front","average":{"1min":17.652999999999999,"5min":16.655000000000001,"15min":19.274000000000001},"min":{"1min":12.808999999999999,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":25.870000000000001,"5min":75.004000000000005,"15min":143.578},"last":22.074000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.600999999999999,"15min":20.175000000000001},"min":{"1min":8.875,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":37.109999999999999,"5min":50.095999999999997,"15min":141.18799999999999},"last":9.9329999999999998},{"interface":"front","average":{"1min":19.908999999999999,"5min":19.231999999999999,"15min":23.358000000000001},"min":{"1min":13.304,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":44.314,"5min":50.887999999999998,"15min":161.32900000000001},"last":26.838000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.367000000000001,"5min":12.712,"15min":14.266999999999999},"min":{"1min":9.8480000000000008,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":30.379999999999999,"5min":30.379999999999999,"15min":74.079999999999998},"last":12.441000000000001},{"interface":"front","average":{"1min":13.227,"5min":14.395,"15min":15.266},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":29.75,"5min":29.75,"15min":58.613999999999997},"last":19.117000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.751000000000001,"5min":14.045999999999999,"15min":16.658999999999999},"min":{"1min":8.8949999999999996,"5min":7.383,"15min":7.383},"max":{"1min":48.072000000000003,"5min":48.072000000000003,"15min":71.757000000000005},"last":8.5549999999999997},{"interface":"front","average":{"1min":22.440000000000001,"5min":19.170000000000002,"15min":20.898},"min":{"1min":11.635999999999999,"5min":9.532,"15min":9.532},"max":{"1min":49.463999999999999,"5min":49.463999999999999,"15min":80.119},"last":19.684000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.024000000000001,"5min":15.693,"15min":18.390999999999998},"min":{"1min":9.1289999999999996,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":27.899000000000001,"5min":29.869,"15min":81.247},"last":22.655999999999999},{"interface":"front","average":{"1min":19.131,"5min":17.087,"15min":18.420999999999999},"min":{"1min":11.939,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":32.530000000000001,"5min":32.530000000000001,"15min":70.989000000000004},"last":28.539999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476818,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":8.5090000000000003,"5min":10.337,"15min":10.819000000000001},"min":{"1min":4.423,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":18.408999999999999,"5min":48.792000000000002,"15min":48.792000000000002},"last":7.1440000000000001},{"interface":"front","average":{"1min":10.489000000000001,"5min":10.949999999999999,"15min":10.488},"min":{"1min":5.6609999999999996,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":19.529,"5min":28.616,"15min":28.616},"last":15.475}]},{"osd":1,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":12.657,"5min":14.374000000000001,"15min":22.675999999999998},"min":{"1min":7.4779999999999998,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":17.920000000000002,"5min":81.552999999999997,"15min":165.76400000000001},"last":7.9619999999999997},{"interface":"front","average":{"1min":11.739000000000001,"5min":13.801,"15min":23.318999999999999},"min":{"1min":7.0460000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":25.146999999999998,"5min":82.995000000000005,"15min":190.34200000000001},"last":31.898}]},{"osd":2,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":13.305999999999999,"5min":15.49,"15min":17.448},"min":{"1min":7.7279999999999998,"5min":7.7279999999999998,"15min":7.7279999999999998},"max":{"1min":21.327000000000002,"5min":42.881999999999998,"15min":42.881999999999998},"last":30.076000000000001},{"interface":"front","average":{"1min":14.321,"5min":15.756,"15min":16.718},"min":{"1min":9.7870000000000008,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":20.465,"5min":71.721000000000004,"15min":71.721000000000004},"last":16.887}]},{"osd":4,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":16.334,"5min":17.201000000000001,"15min":19.48},"min":{"1min":8.6609999999999996,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":33.996000000000002,"5min":44.091000000000001,"15min":44.091000000000001},"last":42.607999999999997},{"interface":"front","average":{"1min":10.779999999999999,"5min":11.087,"15min":11.438000000000001},"min":{"1min":7.4119999999999999,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":36.658000000000001,"5min":45.231000000000002,"15min":45.231000000000002},"last":29.103000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:30:28 2023","interfaces":[{"interface":"back","average":{"1min":13.082000000000001,"5min":15.159000000000001,"15min":18.158999999999999},"min":{"1min":6.8769999999999998,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":26.584,"5min":57.375999999999998,"15min":57.375999999999998},"last":16.117999999999999},{"interface":"front","average":{"1min":16.992000000000001,"5min":15.164,"15min":18.920000000000002},"min":{"1min":8.5969999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":27.321000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":14.337}]},{"osd":6,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":13.994,"5min":16.338999999999999,"15min":24.870999999999999},"min":{"1min":9.9049999999999994,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":22.645,"5min":41.771000000000001,"15min":169.05199999999999},"last":28.030999999999999},{"interface":"front","average":{"1min":16.689,"5min":15.561,"15min":15.516},"min":{"1min":9.6259999999999994,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":25.077000000000002,"5min":40.128999999999998,"15min":40.128999999999998},"last":30.756}]},{"osd":7,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":18.183,"5min":19.457999999999998,"15min":18.475999999999999},"min":{"1min":13.298999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":24.167999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":15.087999999999999},{"interface":"front","average":{"1min":18.882000000000001,"5min":20.048999999999999,"15min":26.425999999999998},"min":{"1min":11.368,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":28.706,"5min":55.331000000000003,"15min":170.232},"last":47.518999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476816,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":10.113,"5min":11.311,"15min":22.309000000000001},"min":{"1min":5.6219999999999999,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":24.774999999999999,"5min":91.290999999999997,"15min":258.66199999999998},"last":8.5530000000000008},{"interface":"front","average":{"1min":10.486000000000001,"5min":11.595000000000001,"15min":19.727},"min":{"1min":5.6580000000000004,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":23.321999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":11.675000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":11.568,"5min":12.488,"15min":15.036},"min":{"1min":5.3419999999999996,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":23.634,"5min":31.689,"15min":45.936999999999998},"last":10.763},{"interface":"front","average":{"1min":14.749000000000001,"5min":12.846,"15min":14.173999999999999},"min":{"1min":8.1359999999999992,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":24.257000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":11.034000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":10.035,"5min":10.699,"15min":16.449999999999999},"min":{"1min":8.1059999999999999,"5min":6.1950000000000003,"15min":6.1950000000000003},"max":{"1min":13.409000000000001,"5min":24.704000000000001,"15min":183.38900000000001},"last":15.268000000000001},{"interface":"front","average":{"1min":14.807,"5min":14.911,"15min":20.347999999999999},"min":{"1min":9.9809999999999999,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":24.312000000000001,"5min":24.317,"15min":189.374},"last":13.912000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":14.207000000000001,"5min":13.550000000000001,"15min":19.155999999999999},"min":{"1min":10.801,"5min":6.9210000000000003,"15min":6.9210000000000003},"max":{"1min":21.907,"5min":31.632000000000001,"15min":186.01400000000001},"last":17.907},{"interface":"front","average":{"1min":15.428000000000001,"5min":14.722,"15min":18.317},"min":{"1min":7.391,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":24.452999999999999,"5min":36.784999999999997,"15min":167.67599999999999},"last":8.2309999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":20.239000000000001,"5min":17.667999999999999,"15min":24.262},"min":{"1min":9.2379999999999995,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":65.251999999999995,"5min":65.251999999999995,"15min":215.80500000000001},"last":14.926},{"interface":"front","average":{"1min":18.771999999999998,"5min":16.721,"15min":23.794},"min":{"1min":12.268000000000001,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":34.042000000000002,"5min":34.042000000000002,"15min":187.863},"last":13.65}]},{"osd":6,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":24.710999999999999,"5min":17.227,"15min":23.917000000000002},"min":{"1min":10.429,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":66.673000000000002,"5min":66.673000000000002,"15min":228.81200000000001},"last":16.456},{"interface":"front","average":{"1min":19.091999999999999,"5min":18.872,"15min":26.523},"min":{"1min":9.5820000000000007,"5min":9.5820000000000007,"15min":9.3710000000000004},"max":{"1min":45.685000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":10.266999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":20.428999999999998,"5min":18.739999999999998,"15min":24.398},"min":{"1min":11.984,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":29.34,"5min":35.898000000000003,"15min":170.43600000000001},"last":15.923999999999999},{"interface":"front","average":{"1min":18.663,"5min":17.48,"15min":21.518000000000001},"min":{"1min":10.481,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":32.140000000000001,"5min":32.140000000000001,"15min":157.00200000000001},"last":18.736000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476817,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":9.282,"5min":10.577,"15min":10.654999999999999},"min":{"1min":6.0270000000000001,"5min":4.5149999999999997,"15min":4.5149999999999997},"max":{"1min":18.388999999999999,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.774},{"interface":"front","average":{"1min":9.4459999999999997,"5min":11.286,"15min":11.699999999999999},"min":{"1min":6.0250000000000004,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":16.806999999999999,"5min":57.805,"15min":57.805},"last":6.2939999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":10.903,"5min":10.342000000000001,"15min":16.815000000000001},"min":{"1min":6.8300000000000001,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":17.661000000000001,"5min":32.957000000000001,"15min":127.95399999999999},"last":13.256},{"interface":"front","average":{"1min":11.614000000000001,"5min":12.388999999999999,"15min":14.044},"min":{"1min":6.6929999999999996,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":20.853000000000002,"5min":35.310000000000002,"15min":35.834000000000003},"last":17.321000000000002}]},{"osd":3,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":16.321999999999999,"5min":14.054,"15min":19.783999999999999},"min":{"1min":8.0299999999999994,"5min":6.9530000000000003,"15min":6.9530000000000003},"max":{"1min":43.226999999999997,"5min":43.226999999999997,"15min":99.790999999999997},"last":11.292999999999999},{"interface":"front","average":{"1min":16.652999999999999,"5min":15.691000000000001,"15min":16.373000000000001},"min":{"1min":9.0540000000000003,"5min":6.516,"15min":6.516},"max":{"1min":41.331000000000003,"5min":43.860999999999997,"15min":43.860999999999997},"last":15.961}]},{"osd":4,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":12.659000000000001,"5min":13.193,"15min":15.497},"min":{"1min":8.8780000000000001,"5min":6.8150000000000004,"15min":6.8150000000000004},"max":{"1min":21.021000000000001,"5min":51.802,"15min":51.802},"last":10.644},{"interface":"front","average":{"1min":17.318000000000001,"5min":16.242000000000001,"15min":22.329999999999998},"min":{"1min":9.4009999999999998,"5min":7.6689999999999996,"15min":7.6689999999999996},"max":{"1min":30.931000000000001,"5min":40.000999999999998,"15min":103.705},"last":7.4589999999999996}]},{"osd":5,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":16.969000000000001,"5min":14.619,"15min":17.565000000000001},"min":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":42.204999999999998,"5min":42.204999999999998,"15min":43.847000000000001},"last":12.544},{"interface":"front","average":{"1min":20.196000000000002,"5min":18.030999999999999,"15min":24.908999999999999},"min":{"1min":12.566000000000001,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":40.551000000000002,"5min":40.598999999999997,"15min":98.707999999999998},"last":9.9640000000000004}]},{"osd":6,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":15.388999999999999,"5min":17.584,"15min":19.530000000000001},"min":{"1min":10.585000000000001,"5min":8.7550000000000008,"15min":8.6869999999999994},"max":{"1min":23.635000000000002,"5min":55.631,"15min":55.631},"last":14.061},{"interface":"front","average":{"1min":16.524000000000001,"5min":17.041,"15min":20.547000000000001},"min":{"1min":8.2080000000000002,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":24.529,"5min":54.656999999999996,"15min":54.656999999999996},"last":15.34}]},{"osd":7,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":18.231000000000002,"5min":18.623000000000001,"15min":20.957000000000001},"min":{"1min":11.718999999999999,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":32.253999999999998,"5min":53.314,"15min":53.314},"last":15.065},{"interface":"front","average":{"1min":18.643999999999998,"5min":18.413,"15min":21.658999999999999},"min":{"1min":10.534000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":38.07,"5min":47.578000000000003,"15min":47.578000000000003},"last":18.273}]}]},{"osd":7,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":14.269,"5min":10.869,"15min":14.519},"min":{"1min":4.4470000000000001,"5min":4.431,"15min":4.4249999999999998},"max":{"1min":109.586,"5min":109.586,"15min":131.477},"last":10.369},{"interface":"front","average":{"1min":15.268000000000001,"5min":11.801,"15min":14.923},"min":{"1min":6.5730000000000004,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":89.219999999999999,"5min":89.219999999999999,"15min":129.386},"last":15.948}]},{"osd":1,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":17.631,"5min":12.243,"15min":11.41},"min":{"1min":7.1200000000000001,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":68.385000000000005,"5min":68.385000000000005,"15min":68.385000000000005},"last":15.329000000000001},{"interface":"front","average":{"1min":23.324000000000002,"5min":17.375,"15min":18.783999999999999},"min":{"1min":11.907999999999999,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":77.412999999999997,"5min":77.412999999999997,"15min":91.295000000000002},"last":19.157}]},{"osd":2,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":12.904,"5min":12.074,"15min":16.170000000000002},"min":{"1min":5.1470000000000002,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":30.172999999999998,"5min":46.811,"15min":134.21199999999999},"last":8.9600000000000009},{"interface":"front","average":{"1min":12.067,"5min":13.321999999999999,"15min":14.196999999999999},"min":{"1min":6.8799999999999999,"5min":5.992,"15min":5.992},"max":{"1min":17.074999999999999,"5min":50.450000000000003,"15min":50.450000000000003},"last":21.143000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":13.842000000000001,"5min":14.667999999999999,"15min":18.667999999999999},"min":{"1min":7.5540000000000003,"5min":6.758,"15min":6.758},"max":{"1min":40.790999999999997,"5min":67.840000000000003,"15min":132.56100000000001},"last":14.023},{"interface":"front","average":{"1min":16.870999999999999,"5min":16.372,"15min":19.582000000000001},"min":{"1min":10.417,"5min":9.1150000000000002,"15min":6.5780000000000003},"max":{"1min":38.597000000000001,"5min":66.106999999999999,"15min":124.684},"last":18.201000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:30:26 2023","interfaces":[{"interface":"back","average":{"1min":14.448,"5min":14.148,"15min":13.974},"min":{"1min":6.2409999999999997,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":22.405999999999999,"5min":36.552,"15min":36.552},"last":17.361000000000001},{"interface":"front","average":{"1min":16.242000000000001,"5min":16.524999999999999,"15min":15.909000000000001},"min":{"1min":10.403,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":25.494,"5min":36.908999999999999,"15min":36.908999999999999},"last":20.562000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:30:26 2023","interfaces":[{"interface":"back","average":{"1min":11.68,"5min":12.712999999999999,"15min":18.053999999999998},"min":{"1min":6.6790000000000003,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":27.515000000000001,"5min":27.515000000000001,"15min":125.792},"last":11.942},{"interface":"front","average":{"1min":16.539000000000001,"5min":16.797999999999998,"15min":16.937000000000001},"min":{"1min":9.5640000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":22.021999999999998,"5min":29.338000000000001,"15min":29.338000000000001},"last":22.869}]},{"osd":6,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":20.535,"5min":17.939,"15min":27.425000000000001},"min":{"1min":9.4559999999999995,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":59.167000000000002,"5min":59.167000000000002,"15min":327.29599999999999},"last":21.795000000000002},{"interface":"front","average":{"1min":18.082000000000001,"5min":15.004,"15min":27.614000000000001},"min":{"1min":8.1289999999999996,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":57.106999999999999,"5min":57.106999999999999,"15min":323.82400000000001},"last":13.303000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":770048,"data_stored":726595,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":11.613,"5min":10.550000000000001,"15min":13.574},"min":{"1min":5.2430000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":51.149000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":6.0229999999999997},{"interface":"front","average":{"1min":14.231,"5min":11.051,"15min":15.044},"min":{"1min":6.407,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":50.776000000000003,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.208}]},{"osd":1,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":15.244,"5min":14.02,"15min":16.43},"min":{"1min":7.3220000000000001,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":63.587000000000003,"5min":63.587000000000003,"15min":87.515000000000001},"last":8.8620000000000001},{"interface":"front","average":{"1min":17.096,"5min":14.686,"15min":16.904},"min":{"1min":6.8099999999999996,"5min":6.5549999999999997,"15min":6.21},"max":{"1min":65.349000000000004,"5min":65.349000000000004,"15min":85.832999999999998},"last":12.635999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":15.500999999999999,"5min":15.102,"15min":17.276},"min":{"1min":8.9269999999999996,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":36.207000000000001,"5min":36.207000000000001,"15min":113.943},"last":13.26},{"interface":"front","average":{"1min":10.619999999999999,"5min":11.005000000000001,"15min":13.981},"min":{"1min":5.7300000000000004,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":26.721,"5min":33.826000000000001,"15min":80.004000000000005},"last":8.1010000000000009}]},{"osd":3,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":14.715,"5min":15.676,"15min":19.786000000000001},"min":{"1min":6.4770000000000003,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":23.344000000000001,"5min":87.866,"15min":87.866},"last":15.646000000000001},{"interface":"front","average":{"1min":15.343999999999999,"5min":16.809000000000001,"15min":21.571000000000002},"min":{"1min":8.6069999999999993,"5min":7.8390000000000004,"15min":7.8390000000000004},"max":{"1min":27.131,"5min":89.334000000000003,"15min":91.183000000000007},"last":15.315}]},{"osd":4,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":19.757000000000001,"5min":14.317,"15min":13.263999999999999},"min":{"1min":11.198,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":41.762999999999998,"5min":41.762999999999998,"15min":41.762999999999998},"last":16.324999999999999},{"interface":"front","average":{"1min":14.374000000000001,"5min":15.544,"15min":16.638000000000002},"min":{"1min":6.1589999999999998,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":42.552,"5min":42.552,"15min":59.609000000000002},"last":7.4779999999999998}]},{"osd":6,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":15.319000000000001,"15min":15.853999999999999},"min":{"1min":8.9049999999999994,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":45.054000000000002,"5min":46.561999999999998,"15min":66.765000000000001},"last":9.5340000000000007},{"interface":"front","average":{"1min":17.545000000000002,"5min":18.350999999999999,"15min":20.289000000000001},"min":{"1min":6.6079999999999997,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":45.877000000000002,"5min":45.877000000000002,"15min":71.771000000000001},"last":17.724}]},{"osd":7,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":18.405000000000001,"5min":17.689,"15min":18.661000000000001},"min":{"1min":10.512,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":30.574999999999999,"5min":35.192,"15min":64.677000000000007},"last":16.925000000000001},{"interface":"front","average":{"1min":16.315999999999999,"5min":16.181000000000001,"15min":17.646999999999998},"min":{"1min":7.9960000000000004,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":25.489000000000001,"5min":29.597999999999999,"15min":42.531999999999996},"last":18.114000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":9.6120000000000001,"5min":9.7490000000000006,"15min":9.4559999999999995},"min":{"1min":4.6109999999999998,"5min":4.6109999999999998,"15min":4.5410000000000004},"max":{"1min":16.666,"5min":26.984999999999999,"15min":26.984999999999999},"last":11.946},{"interface":"front","average":{"1min":9.4499999999999993,"5min":10.565,"15min":10.56},"min":{"1min":6.0919999999999996,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":22.981000000000002,"5min":25.934999999999999,"15min":25.934999999999999},"last":14.368}]},{"osd":1,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":11.544,"5min":12.411,"15min":11.784000000000001},"min":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"max":{"1min":22.372,"5min":25.122,"15min":25.122},"last":10.266},{"interface":"front","average":{"1min":14.856999999999999,"5min":13.547000000000001,"15min":13.951000000000001},"min":{"1min":8.2110000000000003,"5min":7.218,"15min":6.609},"max":{"1min":20.957999999999998,"5min":30.751999999999999,"15min":30.751999999999999},"last":13.135999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":16.553999999999998,"5min":14.986000000000001,"15min":12.974},"min":{"1min":8.7919999999999998,"5min":7.4530000000000003,"15min":6.6200000000000001},"max":{"1min":23.042000000000002,"5min":26.809000000000001,"15min":26.809000000000001},"last":12.516},{"interface":"front","average":{"1min":13.771000000000001,"5min":14.698,"15min":15.416},"min":{"1min":8.1609999999999996,"5min":7.907,"15min":7.907},"max":{"1min":27.091000000000001,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.952999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":13.349,"5min":10.861000000000001,"15min":11.130000000000001},"min":{"1min":6.1079999999999997,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":32.804000000000002,"5min":66.722999999999999,"15min":66.722999999999999},"last":8.3439999999999994},{"interface":"front","average":{"1min":16.425999999999998,"5min":16.138999999999999,"15min":16.09},"min":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":24.254000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":18.356999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":18.224,"5min":17.117000000000001,"15min":18.774000000000001},"min":{"1min":10.023,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":50.207000000000001,"5min":50.207000000000001,"15min":64.474999999999994},"last":15.073},{"interface":"front","average":{"1min":15.733000000000001,"5min":14.039,"15min":15.487},"min":{"1min":7.0709999999999997,"5min":6.6440000000000001,"15min":5.7770000000000001},"max":{"1min":48.356000000000002,"5min":48.356000000000002,"15min":75.980000000000004},"last":11.18}]},{"osd":6,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":21.312999999999999,"5min":18.829000000000001,"15min":18.795000000000002},"min":{"1min":10.291,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":60.633000000000003,"5min":60.633000000000003,"15min":60.633000000000003},"last":15.824999999999999},{"interface":"front","average":{"1min":17.771999999999998,"5min":14.288,"15min":14.113},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":48.267000000000003,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.385000000000002}]},{"osd":7,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":17.266999999999999,"5min":16.823,"15min":17.846},"min":{"1min":12.065,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":27.675999999999998,"5min":41.783999999999999,"15min":74.977999999999994},"last":19.757000000000001},{"interface":"front","average":{"1min":21.209,"5min":19.164999999999999,"15min":21.291},"min":{"1min":12.215999999999999,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":39.973999999999997,"5min":44.450000000000003,"15min":83.138000000000005},"last":20.414999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:30:39.301 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [7, 0] 2023-10-30T06:30:39.302 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '7', '0'] 2023-10-30T06:30:39.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 7 0 2023-10-30T06:30:39.306 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:39.306 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:39.309 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:39.309 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:39.352 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:39.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:39.408 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:39.409 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.445 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:39.443+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:39.448 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:39.449 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:39.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:39.504 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:39.505 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.546 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:39.545+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:39.591 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:39.592 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:39.627 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:39.628 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:39.628 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:39.628 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:39.629 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:39.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:39.647 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:39.647+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:39.750 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:39.749+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:39.810 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:39.811 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:39.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:39.811 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:39.811 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.811 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:39.812 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:39.812 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 302.83626053099999, 2023-10-30T06:30:39.812 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:39.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:39.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:39.813 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:39.813 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:39.813 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:39.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:39.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.814 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:39.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:39.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:39.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:39.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:39.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:39.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:39.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:39.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:39.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:39.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:39.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:39.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:39.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:39.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:39.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:39.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:39.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:39.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:39.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:39.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:39.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:39.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:39.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:39.822 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:39.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:39.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.823 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:39.823 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:39.823 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 122.815768207, 2023-10-30T06:30:39.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:39.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:39.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:39.824 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:39.824 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:39.824 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.825 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:39.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:39.826 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:39.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:39.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:39.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:39.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:39.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:39.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:39.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:39.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:39.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:39.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:39.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:39.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:39.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:39.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:39.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:39.833 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:39.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:39.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:39.834 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:39.834 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:39.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:39.877 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:39.876+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:39.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2023-10-30T06:30:39.953 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:39.952+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:39.970 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:39.970 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:39.970 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:39.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:40.053 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:40.051+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:40.153 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:40.153+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:40.166 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:40.166 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:40.166 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:40.167 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:40.167 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:40.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:40.254 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:40.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:40.347 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:40.347 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:40.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:40.347 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:40.348 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:40.355 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:40.354+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:40.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:40.456 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:40.455+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:40.489 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:40.490 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:40.490 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:40.490 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:40.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:40.557 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:40.557+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:40.627 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:40.627 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:40.628 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:40.636 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:40.677 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:40.677+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:40.759 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:40.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:40.858 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:40.858 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:40.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:40.859 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:40.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.859 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:40.860 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:40.860 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 387.52127950200003, 2023-10-30T06:30:40.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:40.860 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.861 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.861 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.861 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:40.862 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.862 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.862 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.863 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:40.863 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.863 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.864 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:40.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.864 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.864 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.865 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:40.865 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:40.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:40.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:40.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:40.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:40.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:40.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:40.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:40.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:40.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:40.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:40.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:40.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:40.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:40.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:40.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:40.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:40.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:40.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:40.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:40.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:40.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:40.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:40.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:40.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:40.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.879 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.880 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.880 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.880 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:40.880 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 385.20012431100002, 2023-10-30T06:30:40.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:40.881 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.881 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.881 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.882 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.882 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:40.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.882 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:40.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:40.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:40.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:40.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:40.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:40.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:40.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:40.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:40.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.893 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.894 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.894 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:40.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:40.894 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.895 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.895 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:40.895 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:40.895 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.895 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:40.896 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:40.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.897 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.898 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.898 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:40.899 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 385.19587975399998, 2023-10-30T06:30:40.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:40.899 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.899 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.899 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.900 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.900 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:40.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.900 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:40.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:40.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:40.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:40.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:40.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:40.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:40.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:40.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:40.908 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.909 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.910 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:40.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:40.910 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.910 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.910 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.911 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:40.911 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:40.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.911 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:40.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:40.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.913 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.913 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.914 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.914 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:40.914 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 385.11042132900002, 2023-10-30T06:30:40.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.915 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:40.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:40.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:40.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:40.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:40.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:40.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:40.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:40.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:40.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:40.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:40.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.924 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.924 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:40.924 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:40.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.925 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.925 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.925 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:40.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:40.925 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:40.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:40.928 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.928 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.929 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.929 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.929 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.84117811300001, 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.930 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.931 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.931 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:40.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.931 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:40.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:40.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.939 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.940 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.941 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.942 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.943 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.944 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.84034670199998, 2023-10-30T06:30:40.945 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.946 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.947 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.948 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.949 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.950 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.951 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:40.952 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:40.953 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:40.954 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:40.955 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:40.956 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.957 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.959 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:40.960 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.83967957499999, 2023-10-30T06:30:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.961 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.962 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.963 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.964 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.965 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.966 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.967 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.968 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.969 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.970 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.971 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:40.972 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:40.973 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:40.974 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.975 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.81740550400002, 2023-10-30T06:30:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:40.976 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:40.977 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:40.978 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.979 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.981 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.982 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.983 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:40.984 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:40.985 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:40.986 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:40.987 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:40.988 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:40.989 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:40.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:40.991 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:40.992 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.81675764800002, 2023-10-30T06:30:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:40.993 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.994 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.995 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.996 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.997 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:40.999 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.000 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.001 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.002 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.003 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:41.004 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:41.005 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:41.006 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.007 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 384.81607157500002, 2023-10-30T06:30:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:41.008 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.009 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.010 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.011 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.012 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:41.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:41.014 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:41.015 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:41.016 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:41.017 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:41.018 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.019 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.020 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.021 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.022 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:41.023 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 383.74518960699999, 2023-10-30T06:30:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.024 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.025 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.026 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.027 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.028 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.029 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.030 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.031 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.032 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.033 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:41.034 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:41.035 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:41.036 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:41.037 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.038 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 383.744525641, 2023-10-30T06:30:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.040 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.041 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.042 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.043 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.044 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.045 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.046 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.047 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.048 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.049 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.050 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.051 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:41.052 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.053 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 383.71607183200001, 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.054 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:41.055 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.056 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.057 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:41.058 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.059 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.060 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.061 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.062 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.063 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.064 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.065 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.066 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.067 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.068 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.37430533899999, 2023-10-30T06:30:41.069 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.070 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.071 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.072 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.073 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.074 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:41.075 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:41.076 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:41.077 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:41.078 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:41.079 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:41.080 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.081 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.082 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.083 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:41.084 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.37355666899998, 2023-10-30T06:30:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.085 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.086 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.087 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.088 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.089 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:41.089 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:41.089 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.089 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.089 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.090 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:41.090 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:41.090 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.090 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.090 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.091 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:41.091 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:41.091 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.091 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.091 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.092 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.093 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:41.094 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:41.094 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.094 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.094 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.094 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:41.095 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:41.095 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.095 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.095 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.095 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:41.096 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:41.096 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.096 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.096 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.096 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:41.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:41.098 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.098 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.098 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:41.098 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:41.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.099 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.099 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.099 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.099 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.37288828599998, 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.100 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.101 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.101 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:41.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:41.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.104 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:41.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:41.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:41.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:41.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:41.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:41.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:41.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:41.108 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.108 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.109 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.110 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:41.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:41.110 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.110 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.110 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.111 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:41.111 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:41.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.111 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.111 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:41.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:41.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.114 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.114 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.115 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.115 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:41.115 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.36303761200003, 2023-10-30T06:30:41.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:41.116 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.116 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.116 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:41.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.118 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.118 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:41.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:41.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:41.120 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:41.121 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:41.121 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.121 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.121 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.121 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:41.122 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:41.123 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.123 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.123 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.123 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:41.123 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:41.124 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.124 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.124 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.124 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:41.124 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:41.125 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.126 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.126 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.126 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:41.126 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:41.126 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.127 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.127 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.127 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:41.127 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:41.127 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.128 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.128 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.128 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:41.128 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:41.128 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.129 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.129 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.129 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:41.129 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:41.129 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.130 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:41.131 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.362363797, 2023-10-30T06:30:41.131 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:41.131 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.131 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.131 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.132 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.132 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:41.132 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.132 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.133 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.133 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.133 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:41.133 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.133 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.134 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.135 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.135 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:41.135 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:41.135 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.135 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.136 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.137 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:41.137 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:41.137 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.137 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.137 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.138 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:41.138 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:41.138 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.138 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.138 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.139 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:41.140 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:41.141 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.141 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.141 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.141 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:41.141 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:41.142 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.142 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.142 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.142 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:41.142 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:41.143 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.144 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.144 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.144 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:41.144 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:41.144 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.145 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:41.146 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.361715414, 2023-10-30T06:30:41.146 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:41.146 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.146 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.146 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.147 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.147 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:41.147 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.147 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.148 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.149 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.149 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:41.149 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.149 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.149 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.150 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.150 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:41.150 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:41.150 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.150 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.151 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.152 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:41.152 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:41.152 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.152 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.152 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.153 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:41.153 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:41.153 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.153 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.153 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.154 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:41.154 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:41.154 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.154 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.154 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.155 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:41.156 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:41.156 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.156 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.156 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.156 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:41.157 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:41.158 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.158 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.158 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.158 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:41.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:41.159 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.159 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.159 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.159 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:41.159 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.160 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 376.35759894799997, 2023-10-30T06:30:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:41.161 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:41.162 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:41.163 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.164 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.164 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.164 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:41.164 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:41.164 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:41.165 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.166 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.166 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:41.166 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:41.166 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:41.166 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.168 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:41.168 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:41.168 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:41.168 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.168 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.169 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:41.170 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:41.170 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:41.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.170 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.170 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:41.171 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:41.171 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:41.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.171 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.171 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:41.172 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:41.172 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:41.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.172 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:41.173 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:41.174 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:41.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:41.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:41.174 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:41.174 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:41.175 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:41.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.175 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:41.176 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:41.176 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:41.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:41.178 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:30:41.179 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.5 pg_upmap mapping to [7,0] 2023-10-30T06:30:41.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2023-10-30T06:30:41.181 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:41.176+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:41.278 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:41.277+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:41.349 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:41.350 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:41.350 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:41.350 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:41.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:41.401 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:41.400+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:41.480 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:41.478+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:41.528 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:41.528 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:41.529 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:41.529 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:41.529 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:41.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:41.582 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:41.581+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:41.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:41.668 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:41.669 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:41.669 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:41.678 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:41.680 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:41.680+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:41.781 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:41.781+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:41.827 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:41.827 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:41.828 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:41.828 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:41.838 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:41.857 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:41.858 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:41.882 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:41.881+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:41.945 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:41.945 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:41.984 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:41.983+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:41.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2023-10-30T06:30:42.003 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:42.003 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:42.003 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:42.003 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:42.005 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:42.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:42.040 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:42.041 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.084 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:42.083+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:42.142 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:42.143 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.194 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:42.194+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:42.202 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:42.202 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:42.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:42.246 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:42.246 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.287 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:42.285+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:42.333 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:42.333 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.346 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:42.346 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:42.358 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:42.399 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:42.398+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:42.427 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.489 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:42.488+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:42.541 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:42.542 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:42.553 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:42.553 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:42.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:42.596 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:42.595+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:42.691 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:42.692 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:42.689+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:42.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:42.792 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:42.791+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:42.866 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:42.867 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:42.876 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:42.917 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:42.916+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:42.994 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:42.994+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:43.000 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2023-10-30T06:30:43.058 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.059 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:43.112 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:43.110+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:43.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:30:43.215 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.216 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.224 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:43.223+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:43.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:43.298 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:43.297+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:43.397 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.397 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.398 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:43.396+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:43.406 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:43.499 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:43.497+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:43.574 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.574 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:43.624 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:43.623+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:43.701 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:43.699+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:43.750 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.751 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:43.802 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:43.800+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:43.903 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:43.902+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:43.926 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:43.926 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:43.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:44.003 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:44.002+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:44.066 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.066 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.076 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:44.077 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:30:44.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2023-10-30T06:30:44.117 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:44.116+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:44.204 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:44.204+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:44.256 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.256 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:44.305 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:44.303+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:44.395 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.395 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:44.406 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:44.404+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:44.507 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:44.505+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:44.535 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.536 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:44.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:44.611 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.492 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.612 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:44.610+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:44.614 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.495 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.651 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.531 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.662 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:10.997 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.667 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:11.001 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.680 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.681 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:44.697 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:11.032 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:44.731 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:44.730+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:44.809 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:44.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:44.861 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:44.861 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:44.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:44.913 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:44.911+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:45.010 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:45.009+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:45.011 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.892 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.012 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:45.013 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.013 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.893 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.026 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:16.906 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.063 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:45.063 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:45.074 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:45.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2023-10-30T06:30:45.108 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:11.443 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.109 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:11.444 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.110 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:45.109+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:45.117 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:11.453 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:45.130 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.211 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:45.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:45.219 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.308 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:45.309 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.312 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:45.311+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:45.323 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:45.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 391.97894318200002, 2023-10-30T06:30:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:45.325 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.326 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.328 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:45.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:45.330 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:45.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:45.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:45.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:45.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:45.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.341 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:45.342 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.65778799100002, 2023-10-30T06:30:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.343 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.345 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.346 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.347 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.349 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.357 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.358 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.65354343400003, 2023-10-30T06:30:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.359 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.360 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:45.363 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:45.364 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:45.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:45.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.374 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:45.375 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.56808500900001, 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.376 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.379 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.380 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.381 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.382 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.387 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.389 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.390 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:45.391 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.29884179300001, 2023-10-30T06:30:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.392 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.394 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.400 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.401 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.402 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.406 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.29801038199997, 2023-10-30T06:30:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.408 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:45.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:45.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:45.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:45.412 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:45.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:45.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:45.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:45.416 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:45.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:45.418 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:45.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:45.420 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:45.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:45.422 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.423 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.29734325499999, 2023-10-30T06:30:45.424 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:45.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:45.431 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:45.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.433 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.433 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:45.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:45.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:45.434 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:45.434 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.435 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.435 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:45.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:45.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.437 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:45.437 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:45.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.438 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:45.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:45.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:45.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:45.439 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.439 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.440 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.440 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.440 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:45.441 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.27506918400002, 2023-10-30T06:30:45.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:45.441 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.441 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.441 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.442 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.442 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:45.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.443 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.443 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.443 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:45.443 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.444 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.444 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.444 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.444 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:45.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:45.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:45.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:45.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:45.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:45.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:45.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:45.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:45.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:45.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:45.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:45.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:45.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:45.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:45.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.454 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:45.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:45.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:45.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:45.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.455 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.456 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.27442132800002, 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.457 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.458 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.458 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:45.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.458 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.459 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:45.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.459 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.461 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.461 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.461 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:45.461 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:45.462 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.462 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.462 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.462 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:45.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:45.463 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.464 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.464 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.464 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:45.464 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:45.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:45.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:45.465 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.466 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:45.466 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:45.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.467 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.467 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:45.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:45.468 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:45.468 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:45.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.469 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.469 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:45.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:45.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:45.472 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:45.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.472 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.473 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:45.473 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:45.473 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 389.27373525500002, 2023-10-30T06:30:45.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.474 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.475 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:45.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.475 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:45.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:45.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:45.478 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:45.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:45.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:45.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:45.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.481 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:45.481 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:45.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.481 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.482 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:45.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:45.482 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.482 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.483 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:45.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:45.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:45.485 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:45.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:45.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:45.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.486 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.486 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.486 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:45.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:45.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.489 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:45.489 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:45.489 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 388.20285328699998, 2023-10-30T06:30:45.489 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:45.490 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.490 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.490 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:45.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:45.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:45.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:45.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:45.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:45.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:45.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:45.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:45.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.498 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.498 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:45.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:45.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.500 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.500 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:45.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:45.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.501 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:45.501 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:45.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.502 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:45.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:45.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:45.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:45.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.504 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:45.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:45.504 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.505 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.505 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.505 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:45.506 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:45.506 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 388.20218932099999, 2023-10-30T06:30:45.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:45.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.506 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.507 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.507 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:45.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.508 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.508 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.508 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:45.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.509 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:45.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:45.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.511 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.511 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:45.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:45.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:45.512 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:45.512 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.513 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.513 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:45.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:45.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.514 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:45.514 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:45.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.515 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.515 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:45.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:45.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:45.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:45.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:45.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:45.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:45.520 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.521 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.521 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.521 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.521 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.521 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:45.522 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:45.522 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 388.17373551200001, 2023-10-30T06:30:45.522 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:45.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.522 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.523 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.523 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:45.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.524 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:45.524 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.525 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.525 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:45.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:45.527 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:45.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.527 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:45.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:45.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.528 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:45.529 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:45.529 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.529 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:45.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:45.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:45.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:45.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:45.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:45.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:45.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:45.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:45.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:45.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:45.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:45.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.83196901899998, 2023-10-30T06:30:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.539 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.539 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.539 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.540 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:45.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.541 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.541 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:45.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.542 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.542 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:45.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:45.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:45.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:45.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:45.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:45.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:45.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:45.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:45.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:45.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:45.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:45.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:45.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:45.552 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.553 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.83122034899998, 2023-10-30T06:30:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.554 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:45.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:45.556 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:45.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:45.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:45.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:45.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:45.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:45.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:45.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:45.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:45.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.570 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.83055196599997, 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.571 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.572 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.575 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.578 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.579 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.581 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.582 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.583 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.584 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.585 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:45.586 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.82070129200002, 2023-10-30T06:30:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.587 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:45.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:45.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:45.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.597 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:45.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:45.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.601 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.602 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.820027477, 2023-10-30T06:30:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.603 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.604 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.606 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:45.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:45.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.609 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.611 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.613 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.617 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.618 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.819379094, 2023-10-30T06:30:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.619 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.620 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:45.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:45.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:45.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:45.632 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:45.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.634 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 380.81526262800003, 2023-10-30T06:30:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:45.635 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:45.636 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:45.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:45.640 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:45.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:45.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:45.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:45.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:45.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:45.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:45.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:45.651 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:45.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:45.653 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:45.653 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.654 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:45.654 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.654 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:45.655 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.655 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:45.656 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:45.657 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:45.651+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:45.753 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:45.751+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:45.837 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:45.837 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:45.838 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:45.838 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:45.846 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:45.891 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:45.889+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:45.960 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:45.959+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:45.974 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:45.974 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:45.974 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:45.975 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:45.975 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:45.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:46.068 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:46.066+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:46.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:30:46.119 INFO:tasks.mon_recovery:adding mon c back in 2023-10-30T06:30:46.120 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-30T06:30:46.120 DEBUG:teuthology.orchestra.run.smithi079:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-10-30T06:30:46.156 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:46.155+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:46.158 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:46.158 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:46.159 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:46.159 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:46.161 INFO:tasks.ceph.mon.c:Started 2023-10-30T06:30:46.161 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-10-30T06:30:46.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:46.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:46.180 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:46.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:46.257 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:46.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:46.340 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:46.340 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:46.341 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:46.341 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:46.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:46.358 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:46.357+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:46.459 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:46.458+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:46.538 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:46.538 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:46.539 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:46.539 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:46.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:46.591 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:46.589+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:46.661 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:46.659+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:46.681 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.682 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:30:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 310.38773754699997, 2023-10-30T06:30:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:30:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:46.683 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:30:46.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:46.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:30:46.686 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:30:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:30:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:30:46.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:30:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:30:46.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:30:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:30:46.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:30:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:30:46.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:30:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:30:46.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:30:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:30:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 130.35556332799999, 2023-10-30T06:30:46.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:46.695 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:30:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:46.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:46.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:30:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:46.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:30:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:30:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:46.700 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:30:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:30:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:46.701 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:30:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:30:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:46.702 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:30:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:30:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:46.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:30:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:30:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:46.704 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:46.705 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:46.705 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:46.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:46.762 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:46.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:46.840 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:46.841 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:46.841 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:46.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:46.892 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:46.890+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:46.964 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:46.962+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:47.016 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:47.017 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:47.017 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:47.017 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:47.017 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:47.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:47.033 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:47.064 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:47.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:47.166 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:47.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:47.177 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:30:47.194 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:47.194 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:47.195 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:47.195 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:47.195 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:47.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:47.267 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:47.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:47.333 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:47.333 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:47.334 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:47.334 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:47.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:47.389 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:47.387+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:47.469 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:47.467+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:47.471 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:47.471 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:47.472 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:47.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:47.570 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:47.569+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:47.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:47.685 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:47.686 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:47.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:47.686 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:47.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.687 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:47.687 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:47.687 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.52935350299998, 2023-10-30T06:30:47.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:47.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:47.688 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:47.688 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:47.688 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:47.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:47.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.689 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:47.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.691 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:47.691 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.692 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.692 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:47.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:47.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.692 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:47.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:47.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:47.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:47.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:47.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:47.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:47.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:47.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:47.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:47.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:47.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:47.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:47.700 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.700 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.700 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.701 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:47.701 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:47.701 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 392.61617963999998, 2023-10-30T06:30:47.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:47.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:47.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:47.703 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:47.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:47.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:47.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.712 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.713 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:47.713 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:47.713 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.95600167100002, 2023-10-30T06:30:47.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:47.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.714 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.716 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:47.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.716 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:47.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:47.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:47.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:47.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:47.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:47.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:47.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:47.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:47.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:47.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:47.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:47.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:47.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:47.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:47.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:47.724 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.724 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.725 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.725 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:47.725 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:47.726 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.880578464, 2023-10-30T06:30:47.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:47.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.726 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:47.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:47.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:47.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:47.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:47.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:47.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:47.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:47.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:47.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:47.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:47.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:47.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:47.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:47.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.736 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.737 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.737 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.737 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:47.737 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.60945533300003, 2023-10-30T06:30:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:47.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:47.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.741 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.741 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:47.741 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:47.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.742 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.742 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.742 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:47.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:47.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:47.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:47.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:47.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:47.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:47.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:47.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:47.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:47.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:47.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:47.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:47.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.750 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.751 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.751 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:47.751 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.57917562300003, 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.753 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.753 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:47.753 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.754 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.754 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:47.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:47.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:47.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:47.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:47.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:47.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:47.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:47.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:47.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:47.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:47.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:47.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:47.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:47.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:47.762 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.762 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.763 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:47.763 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:47.764 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.20526453899998, 2023-10-30T06:30:47.764 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:47.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.764 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:47.766 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:47.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:47.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:47.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:47.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:47.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:47.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:47.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:47.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:47.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:47.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:47.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:47.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:47.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.776 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:47.776 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:47.776 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 391.200765838, 2023-10-30T06:30:47.776 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:47.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:47.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:47.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:47.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:47.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:47.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:47.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:47.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:47.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:47.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:47.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:47.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:47.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.787 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.787 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.787 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.787 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:47.788 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:47.788 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.90916189900003, 2023-10-30T06:30:47.788 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:47.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:47.789 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:47.790 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:47.791 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:47.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.792 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.792 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.792 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:47.792 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:47.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.793 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.793 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.793 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:47.793 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:47.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.795 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.795 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:47.795 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:47.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.796 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.796 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.796 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:47.796 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:47.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.797 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.797 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.797 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:47.797 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:47.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.798 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.798 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.798 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:47.799 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:47.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.799 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.799 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.800 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.800 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:47.800 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:47.800 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.90322458200001, 2023-10-30T06:30:47.800 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.802 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:47.802 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.802 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.802 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.803 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:47.803 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.803 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.803 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.804 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:47.804 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:47.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.804 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.804 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.805 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:47.805 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:47.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.805 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.806 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:47.806 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:47.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.807 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.807 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:47.807 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:47.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.807 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.808 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.808 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:47.808 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:47.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.808 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.809 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.809 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:47.809 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:47.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.809 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.810 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.810 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:47.810 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:47.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.810 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.811 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.811 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:47.811 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:47.811 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.811 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.812 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:47.812 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:47.813 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.90192556699998, 2023-10-30T06:30:47.813 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:47.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.813 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.814 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.814 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.814 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:47.814 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.815 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.815 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.815 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:47.815 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.816 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.816 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.816 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:47.816 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:47.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.817 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.817 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.817 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:47.817 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:47.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.818 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.818 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.818 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:47.818 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:47.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.819 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.819 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.819 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:47.819 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:47.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.820 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.820 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:47.820 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:47.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.821 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.821 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.821 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:47.821 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:47.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.822 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.822 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:47.822 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:47.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.823 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.823 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.823 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:47.823 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:47.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.824 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.825 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:47.825 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:47.825 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.79893675900001, 2023-10-30T06:30:47.825 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:47.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.826 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.826 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.826 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.826 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:47.827 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.827 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.827 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:47.828 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.828 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.828 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.828 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:47.829 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:47.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.829 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.829 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.829 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:47.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:47.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.830 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.830 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.830 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:47.831 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:47.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.831 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.831 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:47.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:47.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.832 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.832 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.832 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.833 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:47.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:47.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:47.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:47.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.836 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.836 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.837 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:47.837 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:47.837 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.52703349199999, 2023-10-30T06:30:47.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:47.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.838 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:47.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:47.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:47.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:47.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:47.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:47.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.844 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:47.844 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:47.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.844 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.844 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.845 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:47.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:47.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.846 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:47.846 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:47.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:47.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:47.848 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.848 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.849 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:47.849 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:47.850 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.51953529999997, 2023-10-30T06:30:47.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:47.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.850 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:47.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:47.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:47.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:47.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:47.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:47.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.857 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.857 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:47.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:47.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.857 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:47.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:47.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:47.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:47.860 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.860 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.861 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.861 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:47.861 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:47.862 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.518874095, 2023-10-30T06:30:47.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:47.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.862 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:47.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:47.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:47.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:47.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:47.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:47.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:47.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.868 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.868 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:47.868 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:47.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.869 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:47.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:47.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:47.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:47.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:47.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:47.872 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.873 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.873 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.873 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:47.874 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:47.874 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.50628825699999, 2023-10-30T06:30:47.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:47.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.875 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:47.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:47.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:47.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:47.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:47.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:47.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.879 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:47.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:47.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.880 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.880 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:47.880 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:47.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:47.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:47.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:47.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:47.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:47.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:47.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:47.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.885 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.885 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.50019604400001, 2023-10-30T06:30:47.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:47.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.887 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.890 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.892 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.893 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.897 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.898 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:47.898 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 390.49920070299999, 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:47.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:47.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:47.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:47.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.904 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.904 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:47.904 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:47.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.905 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.905 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.905 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:47.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:47.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:47.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:47.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:47.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:47.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:47.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:47.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:47.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:47.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:47.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:47.910 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.911 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.911 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.911 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:47.912 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:47.912 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 383.17652813199999, 2023-10-30T06:30:47.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:47.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.913 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.913 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:47.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:47.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.915 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:47.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:47.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.916 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.916 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:47.916 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:47.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:47.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:47.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:47.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:47.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 383.14308664700002, 2023-10-30T06:30:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:47.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:47.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:47.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:47.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:47.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:47.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:47.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.935 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.935 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:47.935 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:47.936 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:47.937 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:47.701+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:47.937 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:47.938 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:47.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:47.939 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:47.936+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:47.988 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:47.988 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.038 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:48.037+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:48.103 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:48.104 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.107 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:48.107 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:48.108 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:48.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:48.139 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:48.137+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:48.191 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:48.191 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.239 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:48.238+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:48.281 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:48.281 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:48.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:48.341 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:48.339+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:48.370 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:48.371 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.443 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:48.442+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:48.487 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:48.488 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.488 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:48.489 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:48.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:48.580 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:48.582 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:48.581+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:48.644 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:48.643+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:48.668 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:48.668 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:48.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:48.749 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:48.747+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:48.843 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:48.843 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:48.846 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:48.844+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:48.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:48.946 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:48.945+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.017 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.026 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:49.070 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:49.068+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:49.148 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:49.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:49.154 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.155 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:49.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:30:49.249 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:49.247+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:49.343 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.343 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.349 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:49.348+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:49.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:49.450 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:49.449+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:49.525 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.525 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:49.576 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:49.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:49.652 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:49.651+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:49.707 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.707 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.717 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:49.761 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:49.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:49.854 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:49.852+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:49.862 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:49.863 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:49.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:49.955 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:49.954+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:50.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:50.057 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.057 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.057 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:50.055+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:50.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:50.156 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:50.155+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:50.254 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.255 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.257 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:50.256+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:50.263 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:50.358 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:50.357+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:50.449 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.450 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:50.459 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:50.457+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:50.560 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:50.559+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:50.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:50.658 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.659 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.661 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:50.660+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:50.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:50.685 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":38,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:50.762 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:50.761+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:50.823 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.823 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:50.859 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:30:50.863 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:50.862+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:50.963 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:50.961+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:50.988 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:50.988 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:50.989 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:50.990 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:50.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:51.064 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.063+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.082 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:51.082 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.165 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.169 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:51.170 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:51.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:51.181 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:51.181 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:51.182 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-10-30T06:30:51.182 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.344467 2023-10-30T06:30:51.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.34446696376874764 2023-10-30T06:30:51.204 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:51.205 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.266 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:51.265+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:51.305 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:51.305 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:51.306 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:51.306 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.314 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:51.367 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:51.365+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:51.400 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:51.400 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.482 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:51.483 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:51.483 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:51.483 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:51.483 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:51.486 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:51.487 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.487 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.486+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.491 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:51.571 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:51.572 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.575 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.574+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.659 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:51.660 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:51.660 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:51.661 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:51.661 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:51.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:51.670 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:51.668+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:51.770 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.769+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.838 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:51.838 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:51.839 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:51.847 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:51.888 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:51.887+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:51.972 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:51.971+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:52.019 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:52.019 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:52.020 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:52.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:52.073 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:52.072+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:52.174 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:52.172+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:52.202 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:52.203 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:52.203 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:52.203 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:52.203 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.203 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:30:52.204 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:52.204 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 400.04619866600001, 2023-10-30T06:30:52.204 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:30:52.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.205 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.206 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.206 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.206 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:52.206 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.207 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.207 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:30:52.207 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:30:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:30:52.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.209 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.209 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:30:52.209 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:30:52.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.210 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.210 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.210 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:30:52.211 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:30:52.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.211 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.211 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.211 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:30:52.212 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:30:52.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.212 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.212 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.212 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:30:52.213 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:30:52.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.213 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:30:52.213 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:30:52.214 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:30:52.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.214 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.214 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.214 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:30:52.215 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:30:52.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.215 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.215 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.215 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:30:52.216 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:30:52.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.216 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.217 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:30:52.217 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:52.217 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 397.13302480300001, 2023-10-30T06:30:52.218 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:30:52.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.218 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.218 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.219 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.219 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:52.219 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.220 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.220 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.220 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:30:52.220 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.221 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.222 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.222 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:30:52.222 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:30:52.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:30:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:30:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.224 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.225 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.225 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:30:52.225 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:30:52.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.225 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.226 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.226 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:30:52.226 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:30:52.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.226 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.227 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.227 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:30:52.227 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:30:52.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.227 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.228 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.228 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:30:52.228 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:30:52.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.228 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.229 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.229 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.229 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:30:52.229 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 396.47284683399999, 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.231 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.231 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:52.231 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.232 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.232 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:30:52.232 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.232 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.233 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.233 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:30:52.233 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:30:52.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.233 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.234 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.234 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:30:52.234 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:30:52.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.234 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.235 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.235 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:30:52.235 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:30:52.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.235 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.236 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.236 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:30:52.236 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:30:52.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.236 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.237 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.238 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:30:52.238 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:30:52.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.238 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.238 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.239 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:30:52.239 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:30:52.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.239 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.239 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.241 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.241 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:30:52.241 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:52.241 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 396.39742362700002, 2023-10-30T06:30:52.241 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:30:52.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.242 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.242 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.242 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.243 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:52.243 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.243 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.244 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:30:52.244 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.244 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.244 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.245 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:30:52.246 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:30:52.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.246 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.246 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.246 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:30:52.247 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:30:52.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.247 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.247 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.247 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:30:52.248 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:30:52.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.248 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.248 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.248 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:30:52.249 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:30:52.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.249 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.249 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.249 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:30:52.250 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:30:52.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.250 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.250 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.250 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:30:52.251 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:30:52.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.252 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.252 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.252 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.253 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:30:52.253 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:52.253 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 396.126300496, 2023-10-30T06:30:52.253 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:30:52.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.254 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.254 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.254 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:52.255 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.255 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.255 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:30:52.256 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:30:52.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.257 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.257 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.257 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:30:52.257 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:30:52.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.258 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.258 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.258 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:30:52.258 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:30:52.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.259 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.259 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.259 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:30:52.259 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:30:52.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.261 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.261 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.261 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:30:52.261 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:30:52.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.262 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.262 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:30:52.262 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:30:52.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.263 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.263 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.263 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:30:52.263 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:30:52.263 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:30:52.264 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:52.265 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 396.096020786, 2023-10-30T06:30:52.265 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:30:52.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.265 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.266 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.266 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.266 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:52.266 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.267 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.267 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:30:52.267 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.268 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.269 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.269 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:30:52.269 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:30:52.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.269 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.270 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.270 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:30:52.270 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:30:52.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.270 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.271 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.271 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:30:52.271 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:30:52.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.272 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.272 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.272 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:30:52.272 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:30:52.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:30:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:30:52.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.274 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.275 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.275 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:30:52.275 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:30:52.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.275 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.276 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:30:52.276 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:52.277 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.72210970200001, 2023-10-30T06:30:52.277 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:30:52.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.277 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.279 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.279 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.279 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:30:52.279 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.280 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.280 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:30:52.280 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:30:52.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.281 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.282 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.282 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:30:52.282 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:30:52.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.282 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.283 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.283 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:30:52.283 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:30:52.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.283 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.284 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.284 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:30:52.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:30:52.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.284 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.285 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.285 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:30:52.285 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:30:52.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.286 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.286 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:30:52.286 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:30:52.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.286 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.287 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.287 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:30:52.287 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:30:52.287 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.287 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:52.288 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.71761100100002, 2023-10-30T06:30:52.289 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:30:52.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.289 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.289 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.290 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.290 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:52.290 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:30:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.291 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.292 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.292 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:30:52.292 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:30:52.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.293 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.293 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:30:52.293 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:30:52.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.293 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.294 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.294 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:30:52.294 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:30:52.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.294 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.295 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.296 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:30:52.296 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:30:52.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.296 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.296 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.297 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:30:52.297 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:30:52.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.297 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.297 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.298 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:30:52.298 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:30:52.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.298 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.298 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:30:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.426007062, 2023-10-30T06:30:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:30:52.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.301 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.301 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.301 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.302 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:52.302 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.302 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.302 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.303 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:30:52.303 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.303 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.303 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.304 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:30:52.305 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:30:52.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.305 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.305 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.305 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:30:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:30:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.306 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:30:52.307 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:30:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:30:52.308 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:30:52.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.309 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.309 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.309 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:30:52.309 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:30:52.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.310 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.310 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.310 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:30:52.310 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:30:52.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.311 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.311 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.42006974499998, 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:30:52.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.313 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.313 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.313 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:52.314 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.314 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.314 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.314 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:30:52.315 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.315 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.315 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.315 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:30:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:30:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:30:52.317 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:30:52.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.317 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.317 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.317 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:30:52.318 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:30:52.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.318 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.318 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:30:52.319 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:30:52.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.319 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.319 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.319 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:30:52.320 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:30:52.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.320 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.320 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.320 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:30:52.321 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:30:52.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.322 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.322 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.322 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:30:52.322 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:30:52.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.323 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.323 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.323 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.324 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:30:52.324 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:52.324 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.41877073000001, 2023-10-30T06:30:52.324 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:30:52.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.325 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.325 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.325 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.325 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.326 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:30:52.327 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:30:52.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.328 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.328 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.328 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:30:52.328 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:30:52.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.329 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.329 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:30:52.329 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:30:52.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.330 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.330 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.330 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:30:52.330 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:30:52.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.331 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.331 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.331 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:30:52.331 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:30:52.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.333 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.333 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.333 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:30:52.333 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:30:52.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.334 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.334 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.334 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:30:52.334 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:30:52.334 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.335 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.335 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.335 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.31578192199999, 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.336 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.337 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.337 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.337 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:52.337 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.338 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.338 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.338 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:30:52.338 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.339 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.339 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:30:52.339 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:30:52.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:30:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:30:52.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:30:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:30:52.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:30:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:30:52.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.343 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:30:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:30:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.344 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:30:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:30:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.345 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:30:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:30:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.346 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:52.347 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.04387865500001, 2023-10-30T06:30:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:30:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:30:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.350 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.351 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:30:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:30:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.352 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:30:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:30:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.353 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:30:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:30:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.354 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:30:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:30:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.355 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:30:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:30:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.356 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:30:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:30:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.357 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:30:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:30:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.358 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:30:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.036380463, 2023-10-30T06:30:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:30:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.360 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.361 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:30:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.362 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:30:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:30:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:30:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:30:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.364 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:30:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:30:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.365 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.366 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:30:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:30:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.367 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:30:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:30:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.368 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:30:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:30:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.369 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:30:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:30:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.370 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.03571925799997, 2023-10-30T06:30:52.371 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:30:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.372 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.373 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:30:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.374 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:30:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:30:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.375 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.376 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:30:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:30:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.377 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:30:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:30:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.378 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:30:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:30:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:30:52.380 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:30:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:30:52.381 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:30:52.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.382 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.382 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.382 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.383 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:30:52.383 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:52.383 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.02313342000002, 2023-10-30T06:30:52.383 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:30:52.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:30:52.385 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:30:52.386 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:30:52.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.387 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.387 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.387 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:30:52.387 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:30:52.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.388 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.388 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:30:52.388 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:30:52.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.389 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.389 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.389 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:30:52.389 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:30:52.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.390 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.390 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:30:52.390 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:30:52.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.391 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.391 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:30:52.391 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:30:52.392 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.392 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.392 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.392 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:30:52.392 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:30:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:30:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:30:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:30:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.01704120699998, 2023-10-30T06:30:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:30:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:30:52.397 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.398 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.398 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:30:52.398 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:30:52.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.399 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.399 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.399 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:30:52.399 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:30:52.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.400 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.400 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.400 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:30:52.400 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:30:52.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.401 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.401 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:30:52.401 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:30:52.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.402 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.402 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.402 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:30:52.402 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:30:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.404 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.404 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.404 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:30:52.404 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:30:52.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.405 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.405 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:30:52.405 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:30:52.405 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.406 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.406 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:30:52.407 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:52.407 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 395.01604586600001, 2023-10-30T06:30:52.407 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:30:52.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.409 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.409 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:30:52.409 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.410 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.410 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:30:52.410 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:30:52.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.411 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.411 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:30:52.411 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:30:52.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.412 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.412 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:30:52.412 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:30:52.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.414 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.414 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:30:52.414 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:30:52.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.415 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.415 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:30:52.415 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:30:52.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.416 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.416 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:30:52.416 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:30:52.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.417 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.417 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:30:52.417 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:30:52.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.417 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.418 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.418 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.418 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:30:52.418 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 387.69337329500001, 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.419 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.421 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.421 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:30:52.421 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.421 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:30:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:30:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.423 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.424 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:30:52.424 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:30:52.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.424 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.424 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.425 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:30:52.425 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:30:52.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.425 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.425 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.426 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:30:52.426 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:30:52.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.426 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.427 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:30:52.428 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:30:52.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.428 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.428 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.428 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:30:52.429 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:30:52.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.429 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.430 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:30:52.430 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:52.430 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 387.65993180999999, 2023-10-30T06:30:52.430 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:30:52.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:52.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:52.431 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:52.431 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.431 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:52.432 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:52.432 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.432 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:52.433 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:30:52.433 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:52.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.433 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.433 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:52.434 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:30:52.435 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:30:52.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.435 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:52.436 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:30:52.437 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:30:52.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.437 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.437 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:52.437 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:30:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:30:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:30:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:30:52.439 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:30:52.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.439 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.439 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:30:52.439 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:30:52.440 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:30:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:52.442 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:52.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:52.444 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:52.442+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:52.557 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:52.556+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:52.571 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:52.572 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:52.572 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:52.580 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:52.665 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:52.663+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:52.746 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:52.746 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:52.746 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:52.746 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:52.747 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:52.747 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:52.745+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:52.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:52.847 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:52.846+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:52.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:30:52.924 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.7 primary-affinity to 0.344467 (8225742) 2023-10-30T06:30:52.925 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:52.925 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:52.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:52.926 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:30:52.926 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:52.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:52.977 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:52.976+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:53.049 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:53.047+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:53.118 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:53.119 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:30:53.119 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:53.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:53.149 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:53.147+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:53.250 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:53.249+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:53.300 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:53.300 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:30:53.300 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:30:53.301 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:30:53.301 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:53.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:53.354 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:53.352+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:53.440 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:30:53.440 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:30:53.441 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:30:53.441 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:30:53.441 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.441 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 316.46623067100001, 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:53.442 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:53.443 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:53.443 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:53.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:30:53.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.443 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:53.444 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.444 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:53.444 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:53.444 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:53.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.445 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:30:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:30:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.447 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:53.447 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:30:53.447 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:30:53.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.448 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:53.448 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:30:53.448 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:30:53.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.448 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.449 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:53.450 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:30:53.450 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:30:53.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.450 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.450 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:53.451 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:30:53.451 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:30:53.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:53.451 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:53.451 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:53.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.452 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:30:53.452 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:53.452 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 136.445738347, 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:30:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:30:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:30:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:30:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:30:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:30:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:30:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:30:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:30:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:30:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:30:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:30:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:30:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:30:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:30:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:30:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:30:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:30:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:30:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:30:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:30:53.462 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:30:53.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:53.464 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:53.462+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:53.565 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:53.563+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:53.598 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:53.598 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:53.598 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:53.598 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:53.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:53.666 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:53.665+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:53.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:53.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:53.780 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:53.778+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:53.794 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:53.794 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:53.795 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:53.795 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:53.795 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:53.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:53.868 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:53.867+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:53.946 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:53.946 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:53.975 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:53.970+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:53.986 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:53.987 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:53.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:54.038 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:54.038 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.070 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:54.069+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:54.124 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:54.124 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.166 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.167 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.171 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:54.170+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:54.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:54.228 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:54.229 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.261 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:30:54.272 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:54.271+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:54.332 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:54.332 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.354 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.355 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:54.408 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:54.406+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:54.433 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:54.433 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.446 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":42,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:54.474 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:54.472+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:54.493 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.494 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:54.518 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:54.519 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.575 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:54.573+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:54.602 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:54.603 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:30:54.652 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.653 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:54.706 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:54.702+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:54.777 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:54.775+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:54.839 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.840 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:54.856 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.628 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:54.899 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:54.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:54.977 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:54.976+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:54.986 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:54.986 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:54.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:55.078 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:55.077+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:55.107 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:08.879 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:30:55.164 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.165 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:55.179 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:55.178+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:55.280 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:55.279+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:55.306 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.306 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:55.381 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:55.380+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:55.444 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.444 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:30:55.482 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:55.481+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:55.580 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.580 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.583 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:55.581+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:55.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:30:55.684 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:55.683+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:55.753 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.754 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.763 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:30:55.805 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:55.804+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:55.885 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:55.884+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:55.934 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:55.935 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:55.943 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:30:55.986 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:55.985+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:56.069 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.069 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.078 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:30:56.119 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:56.118+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:56.188 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:56.187+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:56.246 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.247 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:30:56.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:30:56.325 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:56.323+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:56.391 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:56.389+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.427 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:30:56.522 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:56.520+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:56.593 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:56.591+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:56.607 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.607 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:30:56.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:30:56.701 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:56.699+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:56.786 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.786 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.795 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:56.794+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:56.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:30:56.871 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:56.872 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:30:56.899 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:56.897+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:56.960 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:56.960 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:30:56.965 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:56.965 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:56.974 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:30:57.018 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:57.017+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:57.045 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:30:57.045 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.097 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:57.095+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:57.099 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:57.100 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:57.108 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:30:57.141 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:30:57.141 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.250 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:30:57.250 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.251 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:57.248+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:57.296 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:57.297 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:57.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:30:57.336 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:30:57.336 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.337 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:57.335+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:57.400 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:30:57.401 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:57.399+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:57.428 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:30:57.429 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:30:57.484 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:30:57.485 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:30:57.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:30:57.503 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:57.502+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:57.537 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:30:57.537 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:30:57.602 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:57.600+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:57.641 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:57.641 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:57.642 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:57.642 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:57.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:30:57.735 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:57.733+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:57.816 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:57.814+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:57.830 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:57.831 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:57.831 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:57.831 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:57.831 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:57.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:30:57.924 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:57.923+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:57.941 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 6, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:30:57.941 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:30:57.942 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 4, 5, 7, 6, 3] 2023-10-30T06:30:57.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2023-10-30T06:30:58.005 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:58.003+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:58.013 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:58.013 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:58.013 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:58.014 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:58.014 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:58.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:30:58.107 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:58.105+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:58.159 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":46,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:30:58.191 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:58.191 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:58.191 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:58.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:30:58.247 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:58.244+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:58.306 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:30:58.305+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:30:58.371 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:58.371 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:58.372 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:58.372 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:58.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:30:58.407 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:58.407+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:58.509 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:58.507+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:58.538 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:30:58.534+0000 ff6f640 -1 mon.a@0(leader).osd e121 definitely_dead 0 2023-10-30T06:30:58.610 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:58.609+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:58.613 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:58.613 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:58.613 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:58.613 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:30:58.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.614 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:30:58.614 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:58.614 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 405.27668737800002, 2023-10-30T06:30:58.615 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:30:58.615 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.615 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.615 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.616 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.616 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:30:58.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.616 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.616 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.617 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.618 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:30:58.618 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.618 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.618 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.619 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:30:58.619 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:30:58.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.619 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.619 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.620 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:30:58.620 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:30:58.620 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.620 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.621 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.621 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:30:58.621 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:30:58.621 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.621 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.622 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.622 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:30:58.622 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:30:58.622 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:30:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:30:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.624 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.624 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:30:58.624 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:30:58.624 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:30:58.624 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.625 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.625 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.625 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:30:58.625 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:30:58.625 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.626 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.626 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.626 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:30:58.626 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:30:58.626 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.627 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:30:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:30:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.628 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.629 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.629 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:30:58.629 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:30:58.629 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.629 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.630 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.630 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:30:58.630 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:30:58.630 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.630 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.631 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.631 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:30:58.631 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:30:58.631 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.632 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.632 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.632 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.632 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.95553218700002, 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.633 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.634 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.635 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:30:58.635 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.635 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.635 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.635 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.637 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.637 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:30:58.637 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.637 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.637 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.638 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.638 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:30:58.638 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:30:58.639 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.639 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.639 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.639 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:30:58.640 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:30:58.640 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.640 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.640 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.641 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:30:58.641 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:30:58.641 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.641 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.642 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.642 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:30:58.642 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:30:58.642 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.643 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.643 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.643 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:30:58.643 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:30:58.644 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.644 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.645 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:30:58.645 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:30:58.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.645 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.646 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:30:58.646 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:30:58.646 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.646 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.646 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.647 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:30:58.648 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:30:58.648 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.648 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.648 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.648 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:30:58.649 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:30:58.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.650 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.650 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.650 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.95128763000002, 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:30:58.651 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.652 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.652 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.653 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:58.654 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.654 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.654 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.654 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.655 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:30:58.655 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.655 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.655 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.655 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.656 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:30:58.656 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:30:58.656 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.656 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.656 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.657 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:30:58.657 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:30:58.657 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.657 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.657 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.658 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:30:58.658 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:30:58.658 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.658 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.658 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.659 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:30:58.660 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:30:58.660 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.660 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.660 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.660 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:30:58.661 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:30:58.661 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.661 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.661 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.661 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:30:58.662 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:30:58.662 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.662 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.662 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.662 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:30:58.663 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:30:58.663 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.663 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.663 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.663 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:30:58.664 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:30:58.665 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.665 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.665 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.665 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:30:58.666 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:30:58.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.666 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.666 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.86582920500001, 2023-10-30T06:30:58.667 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:30:58.668 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.668 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.668 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.669 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.669 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:30:58.669 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.669 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.669 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.670 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.670 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:58.670 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.670 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.671 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.671 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.671 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:30:58.671 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.671 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.672 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.672 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.672 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:30:58.672 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:30:58.673 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.673 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.673 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.673 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:30:58.673 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:30:58.674 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.674 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.674 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.674 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:30:58.675 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:30:58.675 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.675 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.676 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.676 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:30:58.676 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:30:58.677 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.677 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.677 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.678 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:30:58.678 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:30:58.678 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.678 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.679 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.680 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:30:58.680 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:30:58.680 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.680 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.680 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.681 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:30:58.681 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:30:58.681 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.681 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.681 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.682 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:30:58.682 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:30:58.682 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.682 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.682 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.683 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:30:58.683 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:30:58.683 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.683 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.683 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.684 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.685 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.685 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.685 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:58.685 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.596585989, 2023-10-30T06:30:58.685 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:30:58.686 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.686 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.686 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.687 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:30:58.687 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.687 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.687 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.687 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.688 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:58.688 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.688 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.688 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.688 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.689 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:30:58.689 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.689 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.689 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.689 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.690 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:30:58.691 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:30:58.691 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.691 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.691 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.691 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:30:58.692 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:30:58.692 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.692 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.692 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.692 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:30:58.693 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:30:58.693 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.693 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.693 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.693 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:30:58.694 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:30:58.695 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.695 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.695 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.695 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:30:58.695 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:30:58.696 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.696 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.696 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.696 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:30:58.696 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:30:58.697 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.697 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.697 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.697 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:30:58.697 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:30:58.698 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.699 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.699 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.699 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:30:58.699 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:30:58.699 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.700 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.700 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.700 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.700 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.700 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.701 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:58.701 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.59575457800003, 2023-10-30T06:30:58.701 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:30:58.701 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.701 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.702 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.703 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.703 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.703 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:58.703 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.704 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.704 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.704 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.704 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:30:58.705 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.705 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.705 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.705 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.705 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:30:58.706 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:30:58.706 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.706 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:30:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:30:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.707 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.708 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.708 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:30:58.708 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:30:58.708 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.709 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.709 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.709 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:30:58.709 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:30:58.709 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:30:58.710 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:30:58.711 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.711 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.711 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.711 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:30:58.712 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:30:58.712 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.712 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.712 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.713 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:30:58.713 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:30:58.713 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.714 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.714 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.714 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:30:58.715 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:30:58.715 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.716 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.716 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.716 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:30:58.717 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:30:58.717 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.717 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.717 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.718 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:30:58.718 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:30:58.718 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.718 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.719 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.719 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:30:58.719 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:30:58.719 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.720 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:58.721 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.59508745099998, 2023-10-30T06:30:58.721 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:30:58.721 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.721 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.721 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.722 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.722 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:30:58.722 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.722 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.723 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.724 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.724 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.724 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:30:58.724 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.724 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.725 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.725 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.725 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:30:58.725 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:30:58.725 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.726 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.727 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.727 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:30:58.727 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:30:58.727 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.727 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.728 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.728 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:30:58.728 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:30:58.728 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.728 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.729 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:30:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:30:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.730 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.731 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:30:58.731 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:30:58.731 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.731 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.731 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.732 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:30:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:30:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.733 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:30:58.734 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:30:58.734 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.734 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.734 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.734 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:30:58.735 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:30:58.735 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.735 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.736 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.736 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.736 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.736 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.736 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:58.737 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.57281338000001, 2023-10-30T06:30:58.737 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:30:58.737 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.737 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.737 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.738 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.738 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:30:58.738 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.738 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.739 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.740 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.740 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:30:58.740 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.740 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.740 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.741 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.741 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:30:58.741 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:30:58.741 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.741 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.742 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.742 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:30:58.742 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:30:58.742 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.742 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.743 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:30:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:30:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.745 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:30:58.745 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:30:58.746 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.746 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.746 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.746 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:30:58.747 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:30:58.748 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.748 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.748 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.748 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:30:58.748 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:30:58.749 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.749 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.749 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.749 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:30:58.749 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:30:58.750 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.750 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.750 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.750 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:30:58.750 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:30:58.751 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.752 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.752 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.752 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.752 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.752 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.753 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:58.753 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.57216552400001, 2023-10-30T06:30:58.753 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:30:58.753 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.753 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.754 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.754 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:30:58.754 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:58.755 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:30:58.756 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:30:58.757 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.758 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.758 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.758 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:30:58.758 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:30:58.758 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.759 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.759 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.759 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:30:58.759 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:30:58.759 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.760 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.760 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.760 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:30:58.760 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:30:58.760 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.761 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.762 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.762 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:30:58.762 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:30:58.762 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.762 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.763 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.763 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:30:58.763 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:30:58.763 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.763 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.764 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.764 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:30:58.764 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:30:58.764 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.764 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.765 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.766 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:30:58.766 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:30:58.766 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.766 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.766 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.767 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.768 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.768 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:30:58.768 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:58.768 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 402.57147945100002, 2023-10-30T06:30:58.768 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:30:58.769 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.769 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.769 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.770 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:30:58.770 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.770 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.770 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.770 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.771 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:58.771 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.771 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.771 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.771 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.772 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:30:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:30:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.773 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:30:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:30:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.774 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:30:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:30:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.775 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:30:58.776 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:30:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:30:58.777 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:30:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:30:58.778 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:30:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:30:58.779 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:30:58.780 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:30:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:30:58.781 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:30:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:30:58.782 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.785 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 401.50059748299998, 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:30:58.786 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.787 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:30:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.788 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.789 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:30:58.790 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:30:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:30:58.791 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:30:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:30:58.792 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:30:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:30:58.793 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:30:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:30:58.794 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:30:58.795 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:30:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:30:58.796 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:30:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:30:58.797 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:30:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:30:58.798 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:30:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:30:58.799 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.800 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:58.801 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 401.49993351699999, 2023-10-30T06:30:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:30:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.802 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:30:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.803 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.804 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:30:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.805 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:30:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:30:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.806 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:30:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:30:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.807 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:30:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:30:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.808 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:30:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:30:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.809 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:30:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:30:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.810 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.811 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:30:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:30:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.812 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:30:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:30:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.813 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:30:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:30:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.814 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.815 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:30:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:30:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.816 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:30:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 401.471479708, 2023-10-30T06:30:58.817 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.818 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:30:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.819 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.820 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:30:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.821 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:30:58.822 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:30:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:30:58.823 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:30:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:30:58.824 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:30:58.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:30:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:30:58.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:30:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:30:58.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:30:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:30:58.828 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:30:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:30:58.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:30:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:30:58.830 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.831 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.832 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.12971321499998, 2023-10-30T06:30:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:30:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.833 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:30:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.834 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:30:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:30:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:30:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:30:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.839 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:30:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:30:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:30:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:30:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.841 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:30:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:30:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.842 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.843 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:30:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:30:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.844 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:30:58.845 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:30:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:30:58.846 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:30:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.847 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.12896454499997, 2023-10-30T06:30:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:30:58.848 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.849 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:30:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.850 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.851 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:30:58.852 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:30:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:30:58.853 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:30:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:30:58.854 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:30:58.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:30:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:30:58.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:30:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:30:58.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:30:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:30:58.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.859 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:30:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:30:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.860 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:30:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:30:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.861 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.862 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:58.863 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.12829616200003, 2023-10-30T06:30:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:30:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.864 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.865 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.866 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:30:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.867 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.868 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:30:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:30:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.869 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:30:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:30:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.870 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:30:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:30:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.871 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:30:58.872 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:30:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:30:58.873 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:30:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:30:58.874 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:30:58.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:30:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:30:58.876 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:30:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:30:58.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.878 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.11844548800002, 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.879 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.880 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.880 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:30:58.880 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.880 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.881 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.881 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.881 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:58.881 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.881 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.882 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.882 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.882 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:30:58.882 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.882 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.883 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.883 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.883 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:30:58.883 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:30:58.883 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.884 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.884 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.884 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:30:58.884 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:30:58.884 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.885 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.886 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.886 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:30:58.886 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:30:58.886 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.886 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.887 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.887 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:30:58.887 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:30:58.887 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.887 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.888 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.888 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:30:58.888 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:30:58.888 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.888 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.889 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.890 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:30:58.890 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:30:58.890 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.890 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.890 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.891 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:30:58.891 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:30:58.891 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.891 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.891 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.892 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:30:58.893 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:30:58.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.893 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.893 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.893 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.894 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.894 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.894 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:58.894 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.11777167299999, 2023-10-30T06:30:58.894 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:30:58.895 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.895 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.895 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.896 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:58.897 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.897 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.897 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.897 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.897 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:30:58.898 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.898 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.898 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.898 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.898 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:30:58.899 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:30:58.899 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.899 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.899 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.899 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:30:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:30:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.900 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:30:58.901 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:30:58.901 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.901 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.901 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.901 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:30:58.902 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:30:58.903 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.903 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.903 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.903 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:30:58.903 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:30:58.904 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.904 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.904 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.904 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:30:58.904 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:30:58.905 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.906 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.906 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.906 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:30:58.906 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:30:58.906 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:30:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:30:58.907 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.908 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.909 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.909 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.909 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.909 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.909 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:58.910 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.11712328999999, 2023-10-30T06:30:58.910 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:30:58.910 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.910 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.910 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.911 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.912 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.912 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:58.912 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.912 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.912 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:30:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.913 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.914 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.915 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:30:58.915 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:30:58.915 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.915 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.915 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.916 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:30:58.916 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:30:58.916 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.916 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.916 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.917 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:30:58.917 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:30:58.917 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.917 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.917 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.918 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:30:58.918 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:30:58.918 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.918 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.918 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.919 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:30:58.920 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:30:58.920 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.920 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.920 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.920 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:30:58.921 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:30:58.921 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.921 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.921 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.921 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:30:58.922 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:30:58.922 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.922 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.922 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.922 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:30:58.923 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:30:58.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.924 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.924 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.924 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.924 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.925 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:30:58.925 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:58.925 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 394.11300682400002, 2023-10-30T06:30:58.925 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:30:58.925 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:30:58.926 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:30:58.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:30:58.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:30:58.926 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:30:58.926 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:58.927 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.928 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.928 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.928 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:30:58.928 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:30:58.928 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:30:58.929 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.929 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.929 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:30:58.929 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:30:58.929 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:30:58.930 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.930 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.930 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:30:58.930 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:30:58.930 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:30:58.931 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.931 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.931 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:30:58.931 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:30:58.931 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:30:58.932 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:30:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:30:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:30:58.933 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.934 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.934 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:30:58.934 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:30:58.934 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:30:58.934 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.935 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.936 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:30:58.936 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:30:58.936 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:30:58.936 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.936 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.937 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:30:58.937 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:30:58.937 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:30:58.937 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.937 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:30:58.938 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:30:58.939 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:30:58.939 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:30:58.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.939 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.939 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.940 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:30:58.940 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:30:58.940 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:58.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:30:58.942 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:58.941+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:59.042 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:59.041+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:59.114 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:59.115 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:59.115 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:59.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:30:59.166 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:30:59.164+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:30:59.244 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:59.242+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:59.291 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:59.292 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:59.292 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:59.292 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:59.292 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:59.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:30:59.345 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:30:59.344+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:30:59.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-10-30T06:30:59.446 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:30:59.445+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:30:59.475 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:59.475 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:30:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:30:59.476 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:30:59.476 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:59.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:30:59.547 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:30:59.546+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:30:59.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:30:59.607 INFO:teuthology.orchestra.run.smithi079.stderr:marked out osd.0. 2023-10-30T06:30:59.624 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:59.625 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:30:59.625 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:59.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:30:59.675 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:59.674+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:30:59.749 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:30:59.747+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:30:59.821 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:30:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:30:59.821 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:30:59.822 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:30:59.822 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:30:59.829 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:30:59.830 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:30:59.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:30:59.924 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:30:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:30:59.927 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:30:59.925+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:30:59.951 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:30:59.950+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:00.048 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:31:00.049 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.052 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:00.050+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:00.097 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:31:00.097 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:31:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:31:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:31:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:00.098 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 323.80333430600001, 2023-10-30T06:31:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:31:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:31:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:00.099 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:31:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:31:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.100 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:00.101 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:00.102 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.103 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:31:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:31:00.104 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:31:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:31:00.105 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:31:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:31:00.106 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.107 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:31:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:31:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:00.108 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:31:00.109 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 143.771160087, 2023-10-30T06:31:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:31:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:31:00.110 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:31:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:31:00.111 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:00.112 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:00.113 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:00.114 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:31:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:31:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.115 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:31:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:31:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.116 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.117 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:31:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:31:00.118 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:31:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:31:00.119 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:31:00.120 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:31:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:00.121 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:00.122 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:00.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:31:00.152 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:31:00.152 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.168 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:00.167+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:00.237 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:31:00.238 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.253 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:00.253+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:00.277 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:00.277 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:00.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:31:00.327 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:31:00.327 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.355 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:00.353+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:00.414 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:31:00.414 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.455 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:00.453+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:00.457 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:00.457 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:00.469 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:31:00.506 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:31:00.507 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:31:00.560 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:00.558+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:00.599 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:00.599 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:00.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:31:00.657 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:00.656+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:00.698 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is set 2023-10-30T06:31:00.738 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:00.738 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:00.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:31:00.761 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:00.759+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:00.879 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:00.859+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:00.907 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:00.907 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:00.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:31:00.960 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:00.959+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:01.062 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:01.061+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:01.082 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.083 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:31:01.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:31:01.188 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:01.187+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:01.265 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:01.263+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:01.271 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.271 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:31:01.367 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:01.366+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.462 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:31:01.503 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:01.503+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:01.566 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:01.565+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:01.634 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.634 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:31:01.667 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:01.666+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:01.768 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:01.767+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:01.811 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.811 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:31:01.837 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":49,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:31:01.892 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:01.890+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:01.946 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:01.946 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:01.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:31:02.001 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:02.000+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:02.072 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:02.071+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:02.081 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:02.081 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:02.091 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:31:02.173 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:02.171+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:02.219 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:02.219 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:02.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:31:02.313 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:02.311+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:02.375 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:02.373+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:02.394 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:02.395 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:02.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:31:02.488 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:02.486+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:02.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:31:02.577 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:02.575+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:02.589 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:02.589 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:02.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:31:02.684 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:02.682+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:02.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-10-30T06:31:02.772 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:02.773 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:02.779 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:02.778+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:02.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:31:02.842 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:31:02.843 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:31:02.879 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:02.879+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:02.980 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:31:02.980 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:31:02.981 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:02.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:02.982 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:02.982 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:31:02.982 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:31:02.983 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:31:02.983 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:02.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:31:03.066 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:31:03.066 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.082 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:03.082+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:03.164 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:31:03.164 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.192 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:03.192 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:31:03.193 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:31:03.193 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:31:03.193 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.193 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e18)", 2023-10-30T06:31:03.194 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:31:03.194 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 409.85193018299998, 2023-10-30T06:31:03.194 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.9136522199999999, 2023-10-30T06:31:03.195 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.195 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.195 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.196 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.196 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 1 2023-10-30T06:31:03.196 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.196 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.197 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.197 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.197 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:31:03.197 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.197 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.198 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.198 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.198 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.281475+0000", 2023-10-30T06:31:03.198 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.198 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.199 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.199 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.199 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.282147+0000", 2023-10-30T06:31:03.199 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067154599999999997 2023-10-30T06:31:03.199 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.200 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.200 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.200 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.287011+0000", 2023-10-30T06:31:03.200 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0048642939999999999 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.291700+0000", 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046883209999999996 2023-10-30T06:31:03.201 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.202 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.202 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.202 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.300768+0000", 2023-10-30T06:31:03.202 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0090681129999999992 2023-10-30T06:31:03.203 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.203 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.203 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.203 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.315716+0000", 2023-10-30T06:31:03.203 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014947976 2023-10-30T06:31:03.204 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.204 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.204 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for peered", 2023-10-30T06:31:03.204 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:13.322764+0000", 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0070479000000000002 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.690767+0000", 2023-10-30T06:31:03.205 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.3680038319999999 2023-10-30T06:31:03.206 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.206 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.206 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.206 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.778773+0000", 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.088006065999999994 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:14.889675+0000", 2023-10-30T06:31:03.207 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11090201700000001 2023-10-30T06:31:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.058479+0000", 2023-10-30T06:31:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.168803973 2023-10-30T06:31:03.209 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.209 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.209 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.209 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.166397+0000", 2023-10-30T06:31:03.209 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107917176 2023-10-30T06:31:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.178982+0000", 2023-10-30T06:31:03.210 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012584883 2023-10-30T06:31:03.211 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.211 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.211 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.211 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.195128+0000", 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016146122999999998 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.212 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:31:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.53077499199998, 2023-10-30T06:31:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.18826795700000001, 2023-10-30T06:31:03.213 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.214 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.214 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.215 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 20 2023-10-30T06:31:03.215 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.215 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.215 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.215 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.216 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:31:03.216 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.216 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.216 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.216 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602631+0000", 2023-10-30T06:31:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.217 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.218 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.602642+0000", 2023-10-30T06:31:03.218 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1056000000000001e-05 2023-10-30T06:31:03.218 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.219 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.219 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.219 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606098+0000", 2023-10-30T06:31:03.219 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003456359 2023-10-30T06:31:03.219 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.220 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.220 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.220 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606287+0000", 2023-10-30T06:31:03.220 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018908400000000001 2023-10-30T06:31:03.220 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606435+0000", 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014778 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.221 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.222 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.222 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.607703+0000", 2023-10-30T06:31:03.222 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012677400000000001 2023-10-30T06:31:03.222 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.609191+0000", 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001488636 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.223 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.224 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.224 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.616080+0000", 2023-10-30T06:31:03.224 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0068890089999999998 2023-10-30T06:31:03.224 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.225 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.225 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.225 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.731766+0000", 2023-10-30T06:31:03.225 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115685871 2023-10-30T06:31:03.225 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.226 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.226 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.226 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.789078+0000", 2023-10-30T06:31:03.226 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.057311622 2023-10-30T06:31:03.227 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.227 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.227 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.227 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790124+0000", 2023-10-30T06:31:03.227 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0010467269999999999 2023-10-30T06:31:03.228 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.228 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.228 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.228 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.790899+0000", 2023-10-30T06:31:03.228 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077407300000000001 2023-10-30T06:31:03.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.229 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.229 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.229 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:31:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.52653043499998, 2023-10-30T06:31:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.190819604, 2023-10-30T06:31:03.231 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.231 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.231 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 21 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.232 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:31:03.233 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.233 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.233 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.233 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.233 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606875+0000", 2023-10-30T06:31:03.234 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.234 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.234 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.234 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.234 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.606885+0000", 2023-10-30T06:31:03.235 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.9790000000000005e-06 2023-10-30T06:31:03.235 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.235 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.235 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.236 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691669+0000", 2023-10-30T06:31:03.236 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084783958000000006 2023-10-30T06:31:03.236 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.236 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.236 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.691840+0000", 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017053200000000001 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.237 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692019+0000", 2023-10-30T06:31:03.238 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017902900000000001 2023-10-30T06:31:03.238 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.238 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.238 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.238 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.698030+0000", 2023-10-30T06:31:03.239 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0060111970000000002 2023-10-30T06:31:03.239 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.239 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.239 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.240 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.699615+0000", 2023-10-30T06:31:03.240 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015846670000000001 2023-10-30T06:31:03.240 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.240 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.240 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.241 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.704652+0000", 2023-10-30T06:31:03.241 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0050371000000000001 2023-10-30T06:31:03.241 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.241 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.242 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.242 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732389+0000", 2023-10-30T06:31:03.242 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.027737642999999999 2023-10-30T06:31:03.242 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.242 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.243 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.243 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.795989+0000", 2023-10-30T06:31:03.243 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.063599227999999994 2023-10-30T06:31:03.243 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.243 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.244 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.244 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.796980+0000", 2023-10-30T06:31:03.244 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00099127299999999998 2023-10-30T06:31:03.244 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.244 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.245 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.245 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.797695+0000", 2023-10-30T06:31:03.245 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000714998 2023-10-30T06:31:03.245 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.245 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.246 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.246 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.246 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.246 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.44107201000003, 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107844106, 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.247 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.248 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.248 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 22 2023-10-30T06:31:03.248 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.248 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.249 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.249 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.249 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:31:03.249 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.249 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.250 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.250 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.250 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692334+0000", 2023-10-30T06:31:03.250 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.251 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.251 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.251 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.251 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692346+0000", 2023-10-30T06:31:03.251 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.2476e-05 2023-10-30T06:31:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692481+0000", 2023-10-30T06:31:03.252 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000134769 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692646+0000", 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016495600000000001 2023-10-30T06:31:03.253 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.254 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.254 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.254 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.692776+0000", 2023-10-30T06:31:03.254 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013068500000000001 2023-10-30T06:31:03.254 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.255 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.255 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.255 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.712259+0000", 2023-10-30T06:31:03.255 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019482814000000001 2023-10-30T06:31:03.256 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.256 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.256 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.256 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.713824+0000", 2023-10-30T06:31:03.256 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001564223 2023-10-30T06:31:03.257 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.257 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.257 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.257 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.717224+0000", 2023-10-30T06:31:03.257 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0034004 2023-10-30T06:31:03.258 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.258 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.258 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.258 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.732496+0000", 2023-10-30T06:31:03.258 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.015272009 2023-10-30T06:31:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.798555+0000", 2023-10-30T06:31:03.259 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.066059336999999996 2023-10-30T06:31:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.260 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.799500+0000", 2023-10-30T06:31:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094497199999999998 2023-10-30T06:31:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.261 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.800178+0000", 2023-10-30T06:31:03.262 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067746500000000003 2023-10-30T06:31:03.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.262 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.262 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.262 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.17182879400002, 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.124428969, 2023-10-30T06:31:03.263 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.264 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.264 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.264 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 30 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:31:03.265 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.266 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.266 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.266 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961577+0000", 2023-10-30T06:31:03.267 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.267 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.267 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.267 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961587+0000", 2023-10-30T06:31:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0458e-05 2023-10-30T06:31:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.268 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961738+0000", 2023-10-30T06:31:03.269 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015073899999999999 2023-10-30T06:31:03.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.269 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.269 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.269 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.961906+0000", 2023-10-30T06:31:03.270 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016774300000000001 2023-10-30T06:31:03.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.270 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.270 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.271 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962050+0000", 2023-10-30T06:31:03.271 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000144146 2023-10-30T06:31:03.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.272 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.272 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.273 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963868+0000", 2023-10-30T06:31:03.273 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00181813 2023-10-30T06:31:03.274 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.274 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.274 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.275 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.965433+0000", 2023-10-30T06:31:03.275 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015645780000000001 2023-10-30T06:31:03.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.276 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.277 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.277 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.967965+0000", 2023-10-30T06:31:03.278 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025320310000000001 2023-10-30T06:31:03.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.279 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.279 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.279 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076219+0000", 2023-10-30T06:31:03.280 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.108254461 2023-10-30T06:31:03.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.281 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.281 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.282 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.084192+0000", 2023-10-30T06:31:03.282 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0079733990000000008 2023-10-30T06:31:03.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.283 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.283 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.284 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.085127+0000", 2023-10-30T06:31:03.284 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00093451999999999999 2023-10-30T06:31:03.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.285 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.286 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.286 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086006+0000", 2023-10-30T06:31:03.286 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00087876399999999998 2023-10-30T06:31:03.287 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.287 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.288 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.289 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.289 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.289 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:31:03.290 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.17099738299999, 2023-10-30T06:31:03.290 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12529285200000001, 2023-10-30T06:31:03.291 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.291 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.291 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.292 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.293 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 31 2023-10-30T06:31:03.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.293 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.294 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.294 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.295 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:31:03.295 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.295 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.296 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.296 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.297 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962408+0000", 2023-10-30T06:31:03.297 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.297 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.298 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.298 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.299 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962418+0000", 2023-10-30T06:31:03.299 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7620000000000007e-06 2023-10-30T06:31:03.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.300 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.300 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.300 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962555+0000", 2023-10-30T06:31:03.300 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000137198 2023-10-30T06:31:03.301 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.301 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.301 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.302 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962699+0000", 2023-10-30T06:31:03.302 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014352500000000001 2023-10-30T06:31:03.302 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.302 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.303 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.303 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.962829+0000", 2023-10-30T06:31:03.303 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013048399999999999 2023-10-30T06:31:03.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.303 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.304 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.304 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.971794+0000", 2023-10-30T06:31:03.304 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089646459999999997 2023-10-30T06:31:03.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.304 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.973338+0000", 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001543883 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.305 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.306 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.975918+0000", 2023-10-30T06:31:03.306 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025802189999999999 2023-10-30T06:31:03.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.306 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.307 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.307 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.076994+0000", 2023-10-30T06:31:03.307 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.101075955 2023-10-30T06:31:03.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.308 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.308 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.308 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086136+0000", 2023-10-30T06:31:03.308 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0091421200000000001 2023-10-30T06:31:03.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.086975+0000", 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083940299999999996 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.309 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.310 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.310 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087701+0000", 2023-10-30T06:31:03.310 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072565699999999999 2023-10-30T06:31:03.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.310 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.311 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.311 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.311 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.311 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.312 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:31:03.312 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.170330256, 2023-10-30T06:31:03.312 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12629067899999999, 2023-10-30T06:31:03.312 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.313 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 32 2023-10-30T06:31:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.314 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:31:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.315 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963075+0000", 2023-10-30T06:31:03.316 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.316 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.316 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.316 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.316 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.963085+0000", 2023-10-30T06:31:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.4760000000000004e-06 2023-10-30T06:31:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.317 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984787+0000", 2023-10-30T06:31:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021702203 2023-10-30T06:31:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.318 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.984940+0000", 2023-10-30T06:31:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015309399999999999 2023-10-30T06:31:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.319 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985081+0000", 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014053899999999999 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.320 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.988365+0000", 2023-10-30T06:31:03.321 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032839399999999999 2023-10-30T06:31:03.321 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.321 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.321 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.321 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.989965+0000", 2023-10-30T06:31:03.322 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016004159999999999 2023-10-30T06:31:03.322 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.322 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.323 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.323 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.993056+0000", 2023-10-30T06:31:03.323 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030905379999999999 2023-10-30T06:31:03.323 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.324 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.324 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.324 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.077756+0000", 2023-10-30T06:31:03.324 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.084700661999999996 2023-10-30T06:31:03.324 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.325 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.325 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.325 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.087821+0000", 2023-10-30T06:31:03.325 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.010064403 2023-10-30T06:31:03.325 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.326 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.326 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.326 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.088652+0000", 2023-10-30T06:31:03.326 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000831714 2023-10-30T06:31:03.326 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.089366+0000", 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071369399999999996 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.327 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.328 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.328 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.328 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.328 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.329 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:31:03.329 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.14805618499997, 2023-10-30T06:31:03.329 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.113234497, 2023-10-30T06:31:03.329 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.329 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 33 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.330 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:31:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.331 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985349+0000", 2023-10-30T06:31:03.332 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.333 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985361+0000", 2023-10-30T06:31:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1235e-05 2023-10-30T06:31:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.334 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985490+0000", 2023-10-30T06:31:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00012905699999999999 2023-10-30T06:31:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.335 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985655+0000", 2023-10-30T06:31:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016574999999999999 2023-10-30T06:31:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.336 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985786+0000", 2023-10-30T06:31:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013076099999999999 2023-10-30T06:31:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.337 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.001948+0000", 2023-10-30T06:31:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016161871000000001 2023-10-30T06:31:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.338 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.003521+0000", 2023-10-30T06:31:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00157271 2023-10-30T06:31:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.339 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.006012+0000", 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0024910750000000001 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.340 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.078509+0000", 2023-10-30T06:31:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.072497158000000006 2023-10-30T06:31:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.341 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097001+0000", 2023-10-30T06:31:03.342 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.018492389000000001 2023-10-30T06:31:03.342 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.342 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.342 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.342 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.097902+0000", 2023-10-30T06:31:03.343 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00090089300000000005 2023-10-30T06:31:03.343 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.343 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.344 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.344 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098584+0000", 2023-10-30T06:31:03.344 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068159799999999995 2023-10-30T06:31:03.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.345 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.345 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.345 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.345 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.346 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.346 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:31:03.346 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.14740832899997, 2023-10-30T06:31:03.346 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.114218983, 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.347 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 34 2023-10-30T06:31:03.348 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.348 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.348 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.348 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.348 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:31:03.349 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.349 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.349 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.349 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.350 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.985997+0000", 2023-10-30T06:31:03.350 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.350 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.350 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.350 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.351 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986006+0000", 2023-10-30T06:31:03.351 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.1030000000000008e-06 2023-10-30T06:31:03.351 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.351 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.351 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.352 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986123+0000", 2023-10-30T06:31:03.352 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00011682 2023-10-30T06:31:03.352 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.352 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.352 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.353 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986314+0000", 2023-10-30T06:31:03.353 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00019065799999999999 2023-10-30T06:31:03.353 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.353 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.354 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.354 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986452+0000", 2023-10-30T06:31:03.354 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013854199999999999 2023-10-30T06:31:03.354 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.355 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.355 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.355 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.013406+0000", 2023-10-30T06:31:03.355 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.026953464999999999 2023-10-30T06:31:03.355 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.356 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.356 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.356 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.014956+0000", 2023-10-30T06:31:03.356 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015498739999999999 2023-10-30T06:31:03.356 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.017862+0000", 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0029065990000000002 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.357 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.358 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.358 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.079287+0000", 2023-10-30T06:31:03.358 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.061424574000000003 2023-10-30T06:31:03.358 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.358 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.359 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.359 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.098713+0000", 2023-10-30T06:31:03.359 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.019425886999999999 2023-10-30T06:31:03.359 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.360 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.360 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.360 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.099546+0000", 2023-10-30T06:31:03.360 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00083297299999999998 2023-10-30T06:31:03.360 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100216+0000", 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00067048800000000001 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.361 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.362 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.362 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.362 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.362 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-10-30T06:31:03.362 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:31:03.363 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 407.14672225599998, 2023-10-30T06:31:03.363 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.115171605, 2023-10-30T06:31:03.363 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.363 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.364 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.364 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 35 2023-10-30T06:31:03.365 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.365 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.365 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.365 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.366 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:31:03.366 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.366 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.366 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.366 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.367 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986683+0000", 2023-10-30T06:31:03.367 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.367 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.367 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.367 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.368 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:15.986692+0000", 2023-10-30T06:31:03.368 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.0899999999999994e-06 2023-10-30T06:31:03.368 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.369 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.369 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.369 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037374+0000", 2023-10-30T06:31:03.369 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050681813999999999 2023-10-30T06:31:03.369 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.370 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.370 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.370 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037539+0000", 2023-10-30T06:31:03.370 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016432099999999999 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.037770+0000", 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00023166899999999999 2023-10-30T06:31:03.371 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.372 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.372 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.372 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.040299+0000", 2023-10-30T06:31:03.372 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025288659999999998 2023-10-30T06:31:03.373 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.373 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.373 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.373 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.041913+0000", 2023-10-30T06:31:03.373 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016138400000000001 2023-10-30T06:31:03.374 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.374 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.374 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.374 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.044941+0000", 2023-10-30T06:31:03.375 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.003027797 2023-10-30T06:31:03.375 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.375 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.375 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.375 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.095678+0000", 2023-10-30T06:31:03.376 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.050736888000000001 2023-10-30T06:31:03.376 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.376 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.376 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.100338+0000", 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0046602420000000002 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.377 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101188+0000", 2023-10-30T06:31:03.378 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000850141 2023-10-30T06:31:03.378 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.378 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.378 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.378 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:16.101855+0000", 2023-10-30T06:31:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00066693700000000004 2023-10-30T06:31:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.379 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.380 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.380 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:31:03.380 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:31:03.380 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 406.07584028799999, 2023-10-30T06:31:03.381 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.120962323, 2023-10-30T06:31:03.381 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.381 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.381 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.382 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.382 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 83 2023-10-30T06:31:03.382 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.382 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.382 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.383 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.384 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057565+0000", 2023-10-30T06:31:03.384 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.384 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.385 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.385 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.385 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057575+0000", 2023-10-30T06:31:03.386 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6649999999999998e-06 2023-10-30T06:31:03.386 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.386 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.386 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.387 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057701+0000", 2023-10-30T06:31:03.387 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000126451 2023-10-30T06:31:03.387 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.388 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.388 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.389 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057864+0000", 2023-10-30T06:31:03.389 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162075 2023-10-30T06:31:03.389 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.390 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.390 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.390 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.057996+0000", 2023-10-30T06:31:03.391 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132237 2023-10-30T06:31:03.391 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.391 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.392 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.392 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.067458+0000", 2023-10-30T06:31:03.392 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0094622070000000003 2023-10-30T06:31:03.393 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.393 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.393 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.394 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.068998+0000", 2023-10-30T06:31:03.394 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015400769999999999 2023-10-30T06:31:03.394 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.395 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.395 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.396 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.071506+0000", 2023-10-30T06:31:03.396 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025082910000000002 2023-10-30T06:31:03.396 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.396 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.396 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.144915+0000", 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.073408181000000003 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.397 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.176795+0000", 2023-10-30T06:31:03.398 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.031880358999999997 2023-10-30T06:31:03.398 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.398 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.398 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.398 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.177777+0000", 2023-10-30T06:31:03.399 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00098219299999999991 2023-10-30T06:31:03.399 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.399 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.399 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.399 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.178528+0000", 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075058700000000004 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.400 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.401 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:31:03.401 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:31:03.401 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 406.075176322, 2023-10-30T06:31:03.401 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12269131799999999, 2023-10-30T06:31:03.402 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.402 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.402 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 84 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.403 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:31:03.404 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.404 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.404 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.404 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.404 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058229+0000", 2023-10-30T06:31:03.405 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.405 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.405 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.405 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.405 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.058239+0000", 2023-10-30T06:31:03.406 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7589999999999992e-06 2023-10-30T06:31:03.406 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.406 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.406 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.407 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078387+0000", 2023-10-30T06:31:03.407 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020147640000000001 2023-10-30T06:31:03.407 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.407 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.408 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.408 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078549+0000", 2023-10-30T06:31:03.408 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016268700000000001 2023-10-30T06:31:03.409 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.409 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.409 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.410 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.078699+0000", 2023-10-30T06:31:03.410 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014943199999999999 2023-10-30T06:31:03.410 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.087383+0000", 2023-10-30T06:31:03.411 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0086839540000000007 2023-10-30T06:31:03.412 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.412 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.412 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.089042+0000", 2023-10-30T06:31:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001658833 2023-10-30T06:31:03.413 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.091694+0000", 2023-10-30T06:31:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026519870000000002 2023-10-30T06:31:03.414 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.415 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.415 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.415 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.166861+0000", 2023-10-30T06:31:03.415 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.075167094000000004 2023-10-30T06:31:03.416 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.416 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.416 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.417 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.179241+0000", 2023-10-30T06:31:03.417 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012380349000000001 2023-10-30T06:31:03.418 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.418 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.418 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.419 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180198+0000", 2023-10-30T06:31:03.419 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00095661599999999998 2023-10-30T06:31:03.419 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.420 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.420 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.420 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.180921+0000", 2023-10-30T06:31:03.421 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072296700000000003 2023-10-30T06:31:03.421 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.421 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.422 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.423 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2023-10-30T06:31:03.423 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:31:03.423 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 406.04672251300002, 2023-10-30T06:31:03.423 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.107877952, 2023-10-30T06:31:03.424 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.424 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.424 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.425 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.425 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 87 2023-10-30T06:31:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:31:03.427 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.427 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.427 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.427 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.427 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086683+0000", 2023-10-30T06:31:03.428 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.428 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.428 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.428 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.428 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.086692+0000", 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.064e-06 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121088+0000", 2023-10-30T06:31:03.429 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.034395588999999997 2023-10-30T06:31:03.430 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.430 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.430 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.431 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121349+0000", 2023-10-30T06:31:03.431 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026142199999999998 2023-10-30T06:31:03.432 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.432 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.432 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.433 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.121608+0000", 2023-10-30T06:31:03.433 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000258659 2023-10-30T06:31:03.433 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.434 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.434 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.434 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.125294+0000", 2023-10-30T06:31:03.435 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0036861569999999998 2023-10-30T06:31:03.435 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.435 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.436 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.436 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.126981+0000", 2023-10-30T06:31:03.436 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001687146 2023-10-30T06:31:03.437 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.437 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.437 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.438 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.129706+0000", 2023-10-30T06:31:03.438 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0027247130000000001 2023-10-30T06:31:03.438 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.438 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.191880+0000", 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062174357 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.440 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193014+0000", 2023-10-30T06:31:03.440 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0011333249999999999 2023-10-30T06:31:03.440 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.440 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.440 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.441 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.193877+0000", 2023-10-30T06:31:03.441 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00086329199999999999 2023-10-30T06:31:03.441 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.441 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.442 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.442 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:17.194561+0000", 2023-10-30T06:31:03.442 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00068422800000000005 2023-10-30T06:31:03.442 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.442 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.443 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.443 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.443 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.443 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.443 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:31:03.444 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.70495602, 2023-10-30T06:31:03.444 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.11791446, 2023-10-30T06:31:03.444 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.444 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.444 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 104 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.445 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.446 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.446 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:31:03.446 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.446 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.446 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.447 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.447 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428450+0000", 2023-10-30T06:31:03.447 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.447 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.447 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.448 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.448 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428461+0000", 2023-10-30T06:31:03.448 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.1354e-05 2023-10-30T06:31:03.448 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.448 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428623+0000", 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000162287 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.449 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.450 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428795+0000", 2023-10-30T06:31:03.450 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00017187600000000001 2023-10-30T06:31:03.450 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.450 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.450 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.451 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.428945+0000", 2023-10-30T06:31:03.451 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014965399999999999 2023-10-30T06:31:03.451 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.451 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.451 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.430755+0000", 2023-10-30T06:31:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0018101320000000001 2023-10-30T06:31:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.452 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.453 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.432336+0000", 2023-10-30T06:31:03.453 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015810659999999999 2023-10-30T06:31:03.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.453 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.453 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.454 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.434929+0000", 2023-10-30T06:31:03.454 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025927480000000002 2023-10-30T06:31:03.454 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.454 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538003+0000", 2023-10-30T06:31:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.103074469 2023-10-30T06:31:03.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.456 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.456 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.541054+0000", 2023-10-30T06:31:03.456 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0030504880000000001 2023-10-30T06:31:03.457 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.457 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.457 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.457 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.545586+0000", 2023-10-30T06:31:03.458 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045325110000000004 2023-10-30T06:31:03.458 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.458 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.458 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.459 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.546364+0000", 2023-10-30T06:31:03.459 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00077787500000000003 2023-10-30T06:31:03.459 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.460 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.460 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.460 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.460 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.461 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.461 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:31:03.461 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.70420734999999, 2023-10-30T06:31:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.119665013, 2023-10-30T06:31:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.462 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.463 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 105 2023-10-30T06:31:03.464 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.464 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.465 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.465 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.465 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:31:03.465 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.466 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.466 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.467 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429198+0000", 2023-10-30T06:31:03.467 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.467 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.468 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.468 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.468 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429216+0000", 2023-10-30T06:31:03.469 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.8227e-05 2023-10-30T06:31:03.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.469 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.470 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.470 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429352+0000", 2023-10-30T06:31:03.470 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000135697 2023-10-30T06:31:03.470 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.471 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.471 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.471 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429505+0000", 2023-10-30T06:31:03.471 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152788 2023-10-30T06:31:03.472 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.472 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.472 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429645+0000", 2023-10-30T06:31:03.473 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013992599999999999 2023-10-30T06:31:03.473 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.473 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.473 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.474 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.451571+0000", 2023-10-30T06:31:03.474 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.021926514000000001 2023-10-30T06:31:03.474 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.474 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.475 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.475 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.453183+0000", 2023-10-30T06:31:03.475 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016112310000000001 2023-10-30T06:31:03.475 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.476 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.476 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.455780+0000", 2023-10-30T06:31:03.476 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025978080000000001 2023-10-30T06:31:03.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.477 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.477 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.477 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.538119+0000", 2023-10-30T06:31:03.477 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.082338964000000001 2023-10-30T06:31:03.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.478 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.478 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.547116+0000", 2023-10-30T06:31:03.478 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0089968099999999992 2023-10-30T06:31:03.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.479 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.479 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548121+0000", 2023-10-30T06:31:03.479 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001005174 2023-10-30T06:31:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.548863+0000", 2023-10-30T06:31:03.480 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00074187400000000001 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.481 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.482 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:31:03.482 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.70353896699999, 2023-10-30T06:31:03.482 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.133947173, 2023-10-30T06:31:03.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 106 2023-10-30T06:31:03.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.484 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.484 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.484 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:31:03.484 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.485 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.485 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.485 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.485 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429867+0000", 2023-10-30T06:31:03.486 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.487 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.487 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.429876+0000", 2023-10-30T06:31:03.487 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.6660000000000003e-06 2023-10-30T06:31:03.488 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.488 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.488 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.488 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439166+0000", 2023-10-30T06:31:03.488 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0092892300000000008 2023-10-30T06:31:03.489 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.489 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.489 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.489 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439333+0000", 2023-10-30T06:31:03.490 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016758699999999999 2023-10-30T06:31:03.490 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.490 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.490 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439480+0000", 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014733199999999999 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.491 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.459941+0000", 2023-10-30T06:31:03.492 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020460832000000002 2023-10-30T06:31:03.492 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.492 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.492 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.492 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.461543+0000", 2023-10-30T06:31:03.493 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0016015999999999999 2023-10-30T06:31:03.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.493 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.493 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.464098+0000", 2023-10-30T06:31:03.494 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025556149999999998 2023-10-30T06:31:03.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.494 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.494 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.549597+0000", 2023-10-30T06:31:03.495 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.085498562 2023-10-30T06:31:03.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.495 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.495 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.495 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.562119+0000", 2023-10-30T06:31:03.496 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.012521648999999999 2023-10-30T06:31:03.496 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.496 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.496 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563061+0000", 2023-10-30T06:31:03.497 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00094192899999999999 2023-10-30T06:31:03.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.497 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.497 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563814+0000", 2023-10-30T06:31:03.498 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00075317099999999996 2023-10-30T06:31:03.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.499 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.499 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:31:03.499 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.69368829299998, 2023-10-30T06:31:03.500 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.12582813900000001, 2023-10-30T06:31:03.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.500 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.500 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 107 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.501 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.502 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:31:03.502 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.502 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.502 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439717+0000", 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.503 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.504 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439728+0000", 2023-10-30T06:31:03.504 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 1.0686999999999999e-05 2023-10-30T06:31:03.504 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.504 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.505 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.505 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.439859+0000", 2023-10-30T06:31:03.505 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013149400000000001 2023-10-30T06:31:03.505 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.505 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.506 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.506 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440025+0000", 2023-10-30T06:31:03.506 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00016525200000000001 2023-10-30T06:31:03.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.507 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.507 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440165+0000", 2023-10-30T06:31:03.508 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00014009100000000001 2023-10-30T06:31:03.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.508 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.509 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.509 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.476135+0000", 2023-10-30T06:31:03.509 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.035970219999999997 2023-10-30T06:31:03.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.510 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.511 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.477785+0000", 2023-10-30T06:31:03.511 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001649955 2023-10-30T06:31:03.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.512 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.512 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.480435+0000", 2023-10-30T06:31:03.513 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026495350000000002 2023-10-30T06:31:03.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.514 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.514 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.550403+0000", 2023-10-30T06:31:03.515 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.069968234000000004 2023-10-30T06:31:03.515 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.516 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.516 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.563946+0000", 2023-10-30T06:31:03.516 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.013543481 2023-10-30T06:31:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.564833+0000", 2023-10-30T06:31:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00088639399999999998 2023-10-30T06:31:03.518 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.518 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.518 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.518 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565545+0000", 2023-10-30T06:31:03.518 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00071279600000000002 2023-10-30T06:31:03.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.519 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.520 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.520 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.520 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:31:03.520 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.69301447800001, 2023-10-30T06:31:03.521 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.126918696, 2023-10-30T06:31:03.521 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.521 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.522 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.522 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 108 2023-10-30T06:31:03.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.523 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.524 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.524 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.524 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:31:03.525 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.525 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.526 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.526 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440391+0000", 2023-10-30T06:31:03.526 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.527 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.527 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.528 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440401+0000", 2023-10-30T06:31:03.528 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.5189999999999998e-06 2023-10-30T06:31:03.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.529 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.529 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.530 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440533+0000", 2023-10-30T06:31:03.530 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000132674 2023-10-30T06:31:03.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.531 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.531 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440684+0000", 2023-10-30T06:31:03.531 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015064899999999999 2023-10-30T06:31:03.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.532 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.532 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.440822+0000", 2023-10-30T06:31:03.532 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013804600000000001 2023-10-30T06:31:03.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.532 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.533 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.533 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.484750+0000", 2023-10-30T06:31:03.533 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.043927753999999999 2023-10-30T06:31:03.533 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.533 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.534 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.534 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.486358+0000", 2023-10-30T06:31:03.534 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001608405 2023-10-30T06:31:03.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.535 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.535 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.488930+0000", 2023-10-30T06:31:03.535 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025717349999999999 2023-10-30T06:31:03.535 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.535 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.551200+0000", 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.062270210999999999 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.536 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.537 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.565673+0000", 2023-10-30T06:31:03.537 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.014473224999999999 2023-10-30T06:31:03.537 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.537 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.538 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.566586+0000", 2023-10-30T06:31:03.538 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00091295299999999999 2023-10-30T06:31:03.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.538 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.539 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.567310+0000", 2023-10-30T06:31:03.539 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00072352500000000004 2023-10-30T06:31:03.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.540 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.540 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.540 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:31:03.540 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.69236609500001, 2023-10-30T06:31:03.541 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.135076838, 2023-10-30T06:31:03.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.541 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.541 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.542 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.542 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 109 2023-10-30T06:31:03.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.542 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.543 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.544 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441040+0000", 2023-10-30T06:31:03.544 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.544 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.441049+0000", 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.8400000000000007e-06 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444647+0000", 2023-10-30T06:31:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0035974309999999999 2023-10-30T06:31:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444799+0000", 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000152552 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.444939+0000", 2023-10-30T06:31:03.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013935799999999999 2023-10-30T06:31:03.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.493208+0000", 2023-10-30T06:31:03.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.048269628000000002 2023-10-30T06:31:03.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.549 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:03.189+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:03.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.494794+0000", 2023-10-30T06:31:03.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0015852990000000001 2023-10-30T06:31:03.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.497350+0000", 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0025566489999999998 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.552008+0000", 2023-10-30T06:31:03.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.054657844999999997 2023-10-30T06:31:03.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.553 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.553 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.572663+0000", 2023-10-30T06:31:03.553 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.020654626999999998 2023-10-30T06:31:03.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.554 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.554 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.554 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.574812+0000", 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0021494040000000002 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576116+0000", 2023-10-30T06:31:03.555 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0013042049999999999 2023-10-30T06:31:03.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.556 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.556 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.557 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4168.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2023-10-30T06:31:03.557 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:31:03.557 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 398.68824962899998, 2023-10-30T06:31:03.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.13399277100000001, 2023-10-30T06:31:03.557 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4168", 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 110 2023-10-30T06:31:03.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.559 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:03.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:03.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:31:03.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:03.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445156+0000", 2023-10-30T06:31:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445166+0000", 2023-10-30T06:31:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 9.7450000000000007e-06 2023-10-30T06:31:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:03.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445297+0000", 2023-10-30T06:31:03.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013173699999999999 2023-10-30T06:31:03.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:03.564 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445448+0000", 2023-10-30T06:31:03.564 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00015101700000000001 2023-10-30T06:31:03.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.564 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.445586+0000", 2023-10-30T06:31:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00013726799999999999 2023-10-30T06:31:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:03.566 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.526508+0000", 2023-10-30T06:31:03.566 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.080922077999999995 2023-10-30T06:31:03.566 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.566 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.566 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "started", 2023-10-30T06:31:03.567 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.528182+0000", 2023-10-30T06:31:03.567 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001674055 2023-10-30T06:31:03.567 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.568 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "waiting for subops from 5", 2023-10-30T06:31:03.568 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.530876+0000", 2023-10-30T06:31:03.568 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0026941180000000001 2023-10-30T06:31:03.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "sub_op_commit_rec", 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.559853+0000", 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.028976564999999999 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.569 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "op_commit", 2023-10-30T06:31:03.570 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.576347+0000", 2023-10-30T06:31:03.570 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.016494392 2023-10-30T06:31:03.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.570 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "commit_sent", 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.577906+0000", 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001559022 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:03.571 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:24:24.579149+0000", 2023-10-30T06:31:03.572 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0012427740000000001 2023-10-30T06:31:03.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.573 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:03.573 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:03.573 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:03.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:31:03.574 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:31:03.574 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.575 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:31:03.575 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.576 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:31:03.576 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.577 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:31:03.578 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:31:03.591 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:03.590+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:03.679 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:03.678+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:03.722 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:03.723 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:31:03.723 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:31:03.723 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:03.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:31:03.780 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:03.779+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:03.852 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is set 2023-10-30T06:31:03.871 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:03.871 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:31:03.871 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:31:03.872 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:31:03.872 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:03.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:31:03.882 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:03.879+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:03.982 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:03.980+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:04.050 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:04.050 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:31:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:31:04.051 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [] 2023-10-30T06:31:04.051 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:04.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:31:04.105 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:04.102+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:04.183 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:04.182+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:04.185 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:31:04.185 INFO:teuthology.orchestra.run.smithi079.stdout: "num_ops": 0 2023-10-30T06:31:04.186 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:04.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:31:04.285 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:04.283+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:04.361 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:04.362 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [], 2023-10-30T06:31:04.362 INFO:teuthology.orchestra.run.smithi079.stdout: "complaint_time": 30, 2023-10-30T06:31:04.362 INFO:teuthology.orchestra.run.smithi079.stdout: "num_blocked_ops": 0 2023-10-30T06:31:04.362 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:04.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:31:04.386 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:04.384+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:04.487 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:04.485+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:04.541 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 20, 2023-10-30T06:31:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 600, 2023-10-30T06:31:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "ops": [ 2023-10-30T06:31:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.542 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:3 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2023-10-30T06:31:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 328.248009685, 2023-10-30T06:31:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0088754550000000008, 2023-10-30T06:31:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:04.543 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:31:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:31:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 3 2023-10-30T06:31:04.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:04.545 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:04.546 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289147+0000", 2023-10-30T06:31:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.547 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289178+0000", 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 3.1021e-05 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:04.548 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289360+0000", 2023-10-30T06:31:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00018197499999999999 2023-10-30T06:31:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:04.549 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.289573+0000", 2023-10-30T06:31:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00021284299999999999 2023-10-30T06:31:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:04.550 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.291985+0000", 2023-10-30T06:31:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.002411935 2023-10-30T06:31:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:04.551 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.296487+0000", 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0045024640000000003 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:25:36.298023+0000", 2023-10-30T06:31:04.552 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.001535217 2023-10-30T06:31:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "description": "osd_op(client.4176.0:9 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2023-10-30T06:31:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "initiated_at": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "age": 148.21583546599999, 2023-10-30T06:31:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0054537140000000001, 2023-10-30T06:31:04.554 INFO:teuthology.orchestra.run.smithi079.stdout: "type_data": { 2023-10-30T06:31:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "flag_point": "reached pg", 2023-10-30T06:31:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_info": { 2023-10-30T06:31:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client": "client.4176", 2023-10-30T06:31:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:04.555 INFO:teuthology.orchestra.run.smithi079.stdout: "tid": 9 2023-10-30T06:31:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "events": [ 2023-10-30T06:31:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "initiated", 2023-10-30T06:31:04.556 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "header_read", 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321321+0000", 2023-10-30T06:31:04.557 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0 2023-10-30T06:31:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "throttled", 2023-10-30T06:31:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321342+0000", 2023-10-30T06:31:04.558 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 2.1189e-05 2023-10-30T06:31:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "all_read", 2023-10-30T06:31:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321569+0000", 2023-10-30T06:31:04.559 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.000226586 2023-10-30T06:31:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "dispatched", 2023-10-30T06:31:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.321896+0000", 2023-10-30T06:31:04.560 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00032649699999999997 2023-10-30T06:31:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "queued_for_pg", 2023-10-30T06:31:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.322157+0000", 2023-10-30T06:31:04.561 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.00026114399999999999 2023-10-30T06:31:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "reached_pg", 2023-10-30T06:31:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.325358+0000", 2023-10-30T06:31:04.562 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0032008869999999999 2023-10-30T06:31:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "event": "done", 2023-10-30T06:31:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "time": "2023-10-30T06:28:36.326775+0000", 2023-10-30T06:31:04.563 INFO:teuthology.orchestra.run.smithi079.stdout: "duration": 0.0014174109999999999 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:04.564 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:04.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:31:04.611 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:04.609+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:04.628 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5, 7, 6, 3] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:31:04.628 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:31:04.666 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:36.546 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.669 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:36.549 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.688 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:04.687+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:04.689 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:36.569 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.712 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:04.713 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:04.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:31:04.732 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:31.067 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.735 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:31.070 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.755 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:31.090 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:04.790 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:04.788+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:04.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:31:04.863 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:04.863 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:04.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:31:04.896 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:04.894+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:04.991 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:04.989+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:05.043 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:31:05.044 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:05.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:31:05.092 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:05.090+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:05.187 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:05.188 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:05.193 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:05.192+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:05.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:31:05.294 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:05.292+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:05.327 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:05.329 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:05.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:31:05.421 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:05.419+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:05.496 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:05.494+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:05.542 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:31:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:31:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.38685767200002, 2023-10-30T06:31:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:31:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:31:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:31:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:31:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:31:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:31:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:31:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:31:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:31:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:31:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:31:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:31:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:31:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:31:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:31:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:31:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:31:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:31:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:31:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:31:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:31:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 410.47368380900002, 2023-10-30T06:31:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:31:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:31:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:31:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:31:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:31:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:31:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:31:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:31:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:31:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:31:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:31:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:31:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.81350584, 2023-10-30T06:31:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:31:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:31:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:31:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:31:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:31:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:31:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:31:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:31:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:31:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:31:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:31:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:31:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:31:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:31:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:31:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:31:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:31:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:31:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.73808263299998, 2023-10-30T06:31:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:31:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:31:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:31:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:31:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:31:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:31:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:31:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:31:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:31:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:31:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:31:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:31:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:31:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:31:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:31:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:31:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:31:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:31:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.46695950200001, 2023-10-30T06:31:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:31:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:31:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:31:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:31:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:31:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:31:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:31:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:31:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:31:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:31:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:31:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:31:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:31:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:31:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.43667979200001, 2023-10-30T06:31:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:31:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:31:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:31:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:31:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:31:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:31:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:31:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:31:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:31:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:31:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:31:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:31:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:31:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:31:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.06276870800002, 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:31:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:31:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:31:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:31:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:31:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:31:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:31:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:31:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:31:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:31:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:31:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:31:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:31:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 409.05827000699998, 2023-10-30T06:31:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:31:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:31:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:31:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:31:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:31:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:31:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:31:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:31:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:31:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:31:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:31:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:31:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:31:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:31:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:31:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:31:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:31:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.76666606800001, 2023-10-30T06:31:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:31:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:31:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:31:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:31:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:31:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:31:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:31:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:31:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:31:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:31:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:31:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:31:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:31:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:31:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:31:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:31:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:31:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:31:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.76072875099999, 2023-10-30T06:31:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:31:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:31:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:31:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:31:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:31:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:31:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:31:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:31:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:31:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:31:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:31:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:31:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:31:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:31:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:31:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:31:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:31:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:31:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.75942973600002, 2023-10-30T06:31:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:31:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:31:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:31:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:31:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:31:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:31:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:31:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:31:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:31:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:31:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:31:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:31:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:31:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.65644092799999, 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:31:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:31:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:31:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:31:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:31:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:31:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:31:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:31:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:31:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:31:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:31:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:31:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:31:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:31:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:31:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.38453766100002, 2023-10-30T06:31:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:31:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:31:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:31:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:31:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:31:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:31:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:31:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:31:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:31:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:31:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:31:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:31:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:31:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:31:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:31:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:31:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:31:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.37703946900001, 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:31:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:31:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:31:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:31:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:31:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:31:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:31:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:31:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:31:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:31:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:31:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:31:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:31:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.37637826399998, 2023-10-30T06:31:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:31:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:31:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:31:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:31:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:31:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:31:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:31:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:31:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:31:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:31:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:31:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:31:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:31:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:31:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:31:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:31:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.36379242599997, 2023-10-30T06:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:31:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:31:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:31:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:31:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:31:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:31:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:31:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:31:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:31:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:31:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:31:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:31:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:31:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:31:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:31:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:31:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:31:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.35770021299999, 2023-10-30T06:31:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:31:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:31:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:31:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:31:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:31:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:31:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:31:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:31:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:31:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:31:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:31:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:31:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:31:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:31:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 408.35670487200002, 2023-10-30T06:31:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:31:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:31:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:31:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:31:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:31:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:31:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:31:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:31:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:31:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:31:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:31:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:31:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:31:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:31:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 401.03403230100002, 2023-10-30T06:31:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:31:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:31:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:31:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:31:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:31:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:31:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:31:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:31:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:31:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:31:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:31:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:31:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:31:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:31:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 401.000590816, 2023-10-30T06:31:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:31:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:31:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:31:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:31:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:31:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:31:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:31:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:31:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:31:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:31:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:31:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:31:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:31:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:31:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:31:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:05.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-10-30T06:31:05.791 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":46,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":53,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:31:05.791 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:31:05.792 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:05.789+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:05.866 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:31:05.891 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:05.890+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:05.969 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:05.970 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:05.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:31:06.019 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:06.018+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:06.093 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:06.092+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:06.106 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:31:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.149 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:06.150 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:06.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:31:06.194 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:06.193+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:06.201 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:31:06.201 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.283 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:31:06.284 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:06.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:31:06.295 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:31:06.296 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:06.294+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:06.296 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.396 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:06.395+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:06.410 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:31:06.411 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.422 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:06.423 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:06.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:31:06.497 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:06.495+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:06.503 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:31:06.503 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.561 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:06.562 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:06.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:31:06.588 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:31:06.588 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.598 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:06.596+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:06.674 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:31:06.675 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.699 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:06.697+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:06.715 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:06.715 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:31:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:31:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 329.740921415, 2023-10-30T06:31:06.716 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:31:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:31:06.717 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:31:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.718 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.719 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.720 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:31:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:31:06.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:31:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:31:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.723 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:31:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:31:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.724 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:31:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:31:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.725 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:31:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:31:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:06.726 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:06.727 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:06.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.727 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.727 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:31:06.728 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:06.728 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 149.720429091, 2023-10-30T06:31:06.729 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:31:06.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:06.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:06.730 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:06.730 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:31:06.730 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:06.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:31:06.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:06.731 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:06.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.733 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.733 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:06.733 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:31:06.733 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:31:06.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.734 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.734 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:06.734 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:31:06.734 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:31:06.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.735 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.735 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:06.735 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:31:06.735 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:31:06.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.737 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:06.737 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:31:06.737 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:31:06.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:06.738 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:06.738 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:06.738 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:31:06.738 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:31:06.739 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:06.739 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:06.739 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:06.739 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:06.739 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:06.740 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:06.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-10-30T06:31:06.769 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:31:06.769 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'false' 2023-10-30T06:31:06.824 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:06.822+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:06.871 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:06.872 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:06.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-10-30T06:31:06.901 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:06.900+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:07.002 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:07.000+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:07.049 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.049 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-10-30T06:31:07.105 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:07.103+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:07.145 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:31:07.192 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.192 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-10-30T06:31:07.243 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:07.241+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:07.304 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:07.303+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:07.368 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.369 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-10-30T06:31:07.405 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:07.404+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:07.504 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.504 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.505 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:07.504+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:07.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-10-30T06:31:07.606 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:07.605+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:07.644 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.644 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-10-30T06:31:07.737 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:07.735+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:07.810 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:07.807+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:07.825 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:07.825 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:07.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-10-30T06:31:07.920 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:07.918+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:08.010 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:08.008+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:08.025 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.025 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-10-30T06:31:08.110 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:08.109+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:08.207 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.208 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.211 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:08.210+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:08.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-10-30T06:31:08.312 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:31:08.310+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:31:08.364 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.364 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-10-30T06:31:08.414 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:31:08.413+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:31:08.514 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:08.513+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:08.544 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.544 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.554 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-10-30T06:31:08.615 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:31:08.614+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:31:08.662 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:22.434 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:08.690 INFO:teuthology.orchestra.run.smithi079.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.690 INFO:teuthology.orchestra.run.smithi079.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-10-30T06:31:08.724 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:22.497 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:08.747 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:08.745+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:08.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-10-30T06:31:08.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-10-30T06:31:08.828 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:31:08.826+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:31:08.833 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:08.833 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:08.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-10-30T06:31:08.927 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:08.926+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:09.002 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:22.774 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:09.015 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:09.015 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:09.019 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:31:09.018+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:31:09.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-10-30T06:31:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:osd.0: {} 2023-10-30T06:31:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:osd.0: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.120 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:09.119+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:09.127 INFO:teuthology.orchestra.run.smithi079.stdout:osd.1: {} 2023-10-30T06:31:09.128 INFO:teuthology.orchestra.run.smithi079.stderr:osd.1: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:31:09.209 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:09.209 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:09.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-10-30T06:31:09.221 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:09.219+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:09.224 INFO:teuthology.orchestra.run.smithi079.stdout:osd.2: {} 2023-10-30T06:31:09.224 INFO:teuthology.orchestra.run.smithi079.stderr:osd.2: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.322 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:09.320+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:09.359 INFO:teuthology.orchestra.run.smithi079.stdout:osd.3: {} 2023-10-30T06:31:09.359 INFO:teuthology.orchestra.run.smithi079.stderr:osd.3: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.394 INFO:teuthology.orchestra.run.smithi112.stdout:ERROR: (22) Invalid argument 2023-10-30T06:31:09.395 INFO:teuthology.orchestra.run.smithi112.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-10-30T06:31:09.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-10-30T06:31:09.445 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:31:09.444+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:31:09.462 INFO:teuthology.orchestra.run.smithi079.stdout:osd.4: {} 2023-10-30T06:31:09.463 INFO:teuthology.orchestra.run.smithi079.stderr:osd.4: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.524 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:31:09.522+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:31:09.567 INFO:teuthology.orchestra.run.smithi079.stdout:osd.5: {} 2023-10-30T06:31:09.568 INFO:teuthology.orchestra.run.smithi079.stderr:osd.5: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.607 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:09.607 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:09.608 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:09.608 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:09.608 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:09.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-10-30T06:31:09.627 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:31:09.625+0000 ac7d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:31:09.629 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5, 7, 6, 3] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-10-30T06:31:09.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-10-30T06:31:09.630 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-10-30T06:31:09.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-10-30T06:31:09.669 INFO:teuthology.orchestra.run.smithi079.stdout:osd.6: {} 2023-10-30T06:31:09.669 INFO:teuthology.orchestra.run.smithi079.stderr:osd.6: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.694 INFO:teuthology.orchestra.run.smithi079.stdout:{"election_epoch":48,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","modified":"2023-10-30T06:22:31.747585Z","created":"2023-10-30T06:22:31.747585Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3301","nonce":0},{"type":"v1","addr":"172.21.15.79:6790","nonce":0}]},"addr":"172.21.15.79:6790/0","public_addr":"172.21.15.79:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-10-30T06:31:09.711 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-10-30T06:31:09.711 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2023-10-30T06:31:09.721 INFO:tasks.thrashosds:joining thrashosds 2023-10-30T06:31:09.773 INFO:teuthology.orchestra.run.smithi079.stdout:osd.7: {} 2023-10-30T06:31:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:osd.7: osd_enable_op_tracker = 'true' 2023-10-30T06:31:09.830 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:09.830 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:09.830 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:09.831 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:31:09.831 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.831 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4168.0:1 1.0 1.4e660c0c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e17)", 2023-10-30T06:31:09.831 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:09.832 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 417.67417092099998, 2023-10-30T06:31:09.832 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.71432199399999996, 2023-10-30T06:31:09.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:09.832 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:09.833 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4168", 2023-10-30T06:31:09.833 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/1139465334", 2023-10-30T06:31:09.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 1 2023-10-30T06:31:09.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.833 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.834 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.834 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:09.834 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.835 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.835 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.835 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.114989+0000", 2023-10-30T06:31:09.835 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.836 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.836 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.836 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.115663+0000", 2023-10-30T06:31:09.836 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00067452500000000004 2023-10-30T06:31:09.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.120386+0000", 2023-10-30T06:31:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047232919999999996 2023-10-30T06:31:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.838 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.838 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.124750+0000", 2023-10-30T06:31:09.838 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0043633939999999996 2023-10-30T06:31:09.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.839 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.839 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.133362+0000", 2023-10-30T06:31:09.839 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0086125139999999999 2023-10-30T06:31:09.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.840 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.840 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.840 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.146467+0000", 2023-10-30T06:31:09.840 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.013104975 2023-10-30T06:31:09.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.841 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.841 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "waiting for peered", 2023-10-30T06:31:09.841 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.153127+0000", 2023-10-30T06:31:09.841 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.006659758 2023-10-30T06:31:09.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.842 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.842 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.842 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.827945+0000", 2023-10-30T06:31:09.842 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.674818213 2023-10-30T06:31:09.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.843 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.843 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.843 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:12.829311+0000", 2023-10-30T06:31:09.843 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013653230000000001 2023-10-30T06:31:09.844 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.844 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.844 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.845 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.845 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:1 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'1, mlcod=0'0)", 2023-10-30T06:31:09.845 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:09.845 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 414.76099705799999, 2023-10-30T06:31:09.845 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.12791234100000001, 2023-10-30T06:31:09.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.846 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.846 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.846 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.847 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028163+0000", 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.848 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.028169+0000", 2023-10-30T06:31:09.849 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 6.7850000000000003e-06 2023-10-30T06:31:09.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.849 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.849 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.030802+0000", 2023-10-30T06:31:09.850 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.002632995 2023-10-30T06:31:09.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.850 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.850 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.031687+0000", 2023-10-30T06:31:09.851 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088483599999999996 2023-10-30T06:31:09.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.851 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.851 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.851 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.032059+0000", 2023-10-30T06:31:09.852 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037161300000000002 2023-10-30T06:31:09.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.852 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.852 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.852 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.033830+0000", 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001771714 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.041123+0000", 2023-10-30T06:31:09.853 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0072920959999999996 2023-10-30T06:31:09.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.854 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.854 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.143240+0000", 2023-10-30T06:31:09.854 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.10211735700000001 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.156075+0000", 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.012834945 2023-10-30T06:31:09.855 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.856 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:20 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'10, mlcod=19'8)", 2023-10-30T06:31:09.857 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:09.857 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 414.10081908900003, 2023-10-30T06:31:09.857 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.098330870000000001, 2023-10-30T06:31:09.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:09.858 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.859 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.859 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.859 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688340+0000", 2023-10-30T06:31:09.859 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.688353+0000", 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.2567e-05 2023-10-30T06:31:09.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.861 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.861 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.861 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.692971+0000", 2023-10-30T06:31:09.861 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0046175979999999997 2023-10-30T06:31:09.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.862 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.862 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.862 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693122+0000", 2023-10-30T06:31:09.862 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151044 2023-10-30T06:31:09.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.863 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.863 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.693229+0000", 2023-10-30T06:31:09.863 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000107074 2023-10-30T06:31:09.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694068+0000", 2023-10-30T06:31:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00083905600000000002 2023-10-30T06:31:09.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.865 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.865 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.865 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.694486+0000", 2023-10-30T06:31:09.865 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00041804100000000002 2023-10-30T06:31:09.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.866 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.866 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.866 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.785982+0000", 2023-10-30T06:31:09.866 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.091496324000000004 2023-10-30T06:31:09.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.786671+0000", 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00068916599999999995 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.867 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:09.868 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 414.025395882, 2023-10-30T06:31:09.869 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025707826999999999, 2023-10-30T06:31:09.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.869 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.870 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.870 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.870 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:09.870 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.871 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.871 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763764+0000", 2023-10-30T06:31:09.871 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.763773+0000", 2023-10-30T06:31:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.3659999999999998e-06 2023-10-30T06:31:09.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.873 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.873 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.873 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764676+0000", 2023-10-30T06:31:09.873 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090270300000000003 2023-10-30T06:31:09.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.874 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.874 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.874 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764827+0000", 2023-10-30T06:31:09.874 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015133599999999999 2023-10-30T06:31:09.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.875 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.875 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.764961+0000", 2023-10-30T06:31:09.875 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134296 2023-10-30T06:31:09.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.876 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.876 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.876 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.769676+0000", 2023-10-30T06:31:09.876 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0047147710000000004 2023-10-30T06:31:09.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.770049+0000", 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00037258300000000001 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.877 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.878 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.878 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.788848+0000", 2023-10-30T06:31:09.878 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018799488 2023-10-30T06:31:09.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.878 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:15.789472+0000", 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062328399999999999 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.880 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.880 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2023-10-30T06:31:09.880 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:09.880 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.75427275099997, 2023-10-30T06:31:09.881 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024542501000000001, 2023-10-30T06:31:09.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.881 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.881 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.883 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.883 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034887+0000", 2023-10-30T06:31:09.883 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.883 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.884 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.884 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.034895+0000", 2023-10-30T06:31:09.884 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 7.7400000000000004e-06 2023-10-30T06:31:09.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.884 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.885 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.885 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040115+0000", 2023-10-30T06:31:09.885 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0052203700000000002 2023-10-30T06:31:09.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.885 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.886 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.886 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040263+0000", 2023-10-30T06:31:09.886 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000148336 2023-10-30T06:31:09.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.887 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.887 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.040376+0000", 2023-10-30T06:31:09.887 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112507 2023-10-30T06:31:09.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.887 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.888 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.888 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.041801+0000", 2023-10-30T06:31:09.888 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014256449999999999 2023-10-30T06:31:09.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.888 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.042190+0000", 2023-10-30T06:31:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00038837799999999999 2023-10-30T06:31:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.889 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.890 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.890 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.058519+0000", 2023-10-30T06:31:09.890 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.016329565000000001 2023-10-30T06:31:09.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.059429+0000", 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090996000000000002 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.891 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.892 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.892 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.892 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'14)", 2023-10-30T06:31:09.892 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.72399304099997, 2023-10-30T06:31:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.02755179, 2023-10-30T06:31:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.893 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.894 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.894 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.894 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:09.894 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.895 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.895 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.895 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065167+0000", 2023-10-30T06:31:09.895 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.896 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.896 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065175+0000", 2023-10-30T06:31:09.896 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.8599999999999999e-06 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.065982+0000", 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00080655400000000002 2023-10-30T06:31:09.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.898 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.898 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.898 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066151+0000", 2023-10-30T06:31:09.898 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00016856600000000001 2023-10-30T06:31:09.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.899 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.899 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.899 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066263+0000", 2023-10-30T06:31:09.899 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000112817 2023-10-30T06:31:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.066991+0000", 2023-10-30T06:31:09.900 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000727592 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.067395+0000", 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000403903 2023-10-30T06:31:09.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.902 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.902 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.902 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.091827+0000", 2023-10-30T06:31:09.902 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024432625999999999 2023-10-30T06:31:09.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.092718+0000", 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089087200000000002 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.903 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.904 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.904 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.904 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:53 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'31, mlcod=19'29)", 2023-10-30T06:31:09.904 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:09.905 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.35008195699999, 2023-10-30T06:31:09.905 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032709370000000001, 2023-10-30T06:31:09.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.905 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.906 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.906 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.906 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:09.906 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439078+0000", 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.907 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439095+0000", 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.6923e-05 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.909 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439254+0000", 2023-10-30T06:31:09.909 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015946299999999999 2023-10-30T06:31:09.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.909 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.909 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.910 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439466+0000", 2023-10-30T06:31:09.910 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021165499999999999 2023-10-30T06:31:09.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.910 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.911 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.439589+0000", 2023-10-30T06:31:09.911 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012350100000000001 2023-10-30T06:31:09.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.911 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.912 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440480+0000", 2023-10-30T06:31:09.912 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00089107499999999996 2023-10-30T06:31:09.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.912 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.912 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.913 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.440882+0000", 2023-10-30T06:31:09.913 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000402161 2023-10-30T06:31:09.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.913 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.914 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.914 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.470691+0000", 2023-10-30T06:31:09.914 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029808329000000001 2023-10-30T06:31:09.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.914 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.471787+0000", 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010962630000000001 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.915 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.916 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.916 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:54 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-10-30T06:31:09.916 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:09.916 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.345583256, 2023-10-30T06:31:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030980081, 2023-10-30T06:31:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.917 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.918 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.919 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.919 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443576+0000", 2023-10-30T06:31:09.919 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.919 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443584+0000", 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0690000000000002e-06 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.920 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443709+0000", 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012454400000000001 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.921 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443865+0000", 2023-10-30T06:31:09.922 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000156329 2023-10-30T06:31:09.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.922 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.922 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.443969+0000", 2023-10-30T06:31:09.923 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010366600000000001 2023-10-30T06:31:09.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.923 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.923 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.923 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447482+0000", 2023-10-30T06:31:09.924 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0035133650000000001 2023-10-30T06:31:09.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.924 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.924 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.924 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.447838+0000", 2023-10-30T06:31:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035581799999999999 2023-10-30T06:31:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.925 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.926 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.473290+0000", 2023-10-30T06:31:09.926 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025451851000000001 2023-10-30T06:31:09.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.926 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.927 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.474556+0000", 2023-10-30T06:31:09.927 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012664390000000001 2023-10-30T06:31:09.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.927 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.927 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.928 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.928 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:67 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'41, mlcod=19'39)", 2023-10-30T06:31:09.928 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:09.928 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.05397931700003, 2023-10-30T06:31:09.929 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.032062536000000003, 2023-10-30T06:31:09.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.929 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.929 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.930 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.930 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:09.930 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735180+0000", 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.931 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.932 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.932 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735189+0000", 2023-10-30T06:31:09.932 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.6810000000000004e-06 2023-10-30T06:31:09.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.932 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735323+0000", 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000134308 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.933 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.934 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735479+0000", 2023-10-30T06:31:09.934 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000155313 2023-10-30T06:31:09.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.934 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.935 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.735597+0000", 2023-10-30T06:31:09.935 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011885899999999999 2023-10-30T06:31:09.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.935 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.935 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.936 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736387+0000", 2023-10-30T06:31:09.936 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078943100000000005 2023-10-30T06:31:09.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.936 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.937 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.736787+0000", 2023-10-30T06:31:09.937 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00040018200000000001 2023-10-30T06:31:09.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.937 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.937 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.938 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.766363+0000", 2023-10-30T06:31:09.938 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029576361999999998 2023-10-30T06:31:09.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.938 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.938 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.767243+0000", 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00087940000000000002 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.940 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.940 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:68 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'42, mlcod=19'39)", 2023-10-30T06:31:09.940 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:09.940 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.04804200000001, 2023-10-30T06:31:09.940 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029422604000000002, 2023-10-30T06:31:09.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.941 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.941 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.941 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.942 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741118+0000", 2023-10-30T06:31:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.943 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741128+0000", 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.0635000000000001e-05 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741252+0000", 2023-10-30T06:31:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012408800000000001 2023-10-30T06:31:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.945 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741643+0000", 2023-10-30T06:31:09.946 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039049199999999999 2023-10-30T06:31:09.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.946 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.946 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.741824+0000", 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018160600000000001 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.947 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.743477+0000", 2023-10-30T06:31:09.948 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001652831 2023-10-30T06:31:09.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.948 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.948 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.948 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.744127+0000", 2023-10-30T06:31:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00065023800000000003 2023-10-30T06:31:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.949 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.769784+0000", 2023-10-30T06:31:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025657012999999999 2023-10-30T06:31:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.950 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770540+0000", 2023-10-30T06:31:09.951 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00075570100000000001 2023-10-30T06:31:09.951 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.951 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.951 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.952 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.952 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:69 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'43, mlcod=19'39)", 2023-10-30T06:31:09.952 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:09.952 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 413.04674298499998, 2023-10-30T06:31:09.952 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028685676, 2023-10-30T06:31:09.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.953 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.953 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.953 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.954 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:09.954 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.954 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.954 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.955 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742417+0000", 2023-10-30T06:31:09.955 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.955 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.955 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.956 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.742426+0000", 2023-10-30T06:31:09.956 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0440000000000001e-06 2023-10-30T06:31:09.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.956 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.957 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.749770+0000", 2023-10-30T06:31:09.957 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0073444929999999997 2023-10-30T06:31:09.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.957 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.957 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.958 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750059+0000", 2023-10-30T06:31:09.958 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00028864500000000003 2023-10-30T06:31:09.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.958 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.958 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.959 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.750188+0000", 2023-10-30T06:31:09.959 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000129593 2023-10-30T06:31:09.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.959 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.959 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.960 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.751722+0000", 2023-10-30T06:31:09.960 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0015331769999999999 2023-10-30T06:31:09.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.960 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.960 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.961 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.752338+0000", 2023-10-30T06:31:09.961 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00061597400000000004 2023-10-30T06:31:09.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.961 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.961 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.962 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.770563+0000", 2023-10-30T06:31:09.962 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.018225605999999998 2023-10-30T06:31:09.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.962 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.771102+0000", 2023-10-30T06:31:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053914399999999997 2023-10-30T06:31:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.963 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.964 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.964 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:72 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'46, mlcod=19'43)", 2023-10-30T06:31:09.964 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:09.964 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.94375417700002, 2023-10-30T06:31:09.965 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029850458999999999, 2023-10-30T06:31:09.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.965 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.965 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.966 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.966 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:09.966 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.967 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.967 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845405+0000", 2023-10-30T06:31:09.967 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.967 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.968 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.968 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.845415+0000", 2023-10-30T06:31:09.968 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.2240000000000001e-06 2023-10-30T06:31:09.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.968 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846255+0000", 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000840335 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.969 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.970 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846437+0000", 2023-10-30T06:31:09.970 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000182227 2023-10-30T06:31:09.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.970 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.970 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.971 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.846554+0000", 2023-10-30T06:31:09.971 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000116332 2023-10-30T06:31:09.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.971 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.972 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847335+0000", 2023-10-30T06:31:09.972 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00078135399999999995 2023-10-30T06:31:09.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.972 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.972 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.973 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.847727+0000", 2023-10-30T06:31:09.973 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039237099999999997 2023-10-30T06:31:09.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.973 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.973 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.974 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.874150+0000", 2023-10-30T06:31:09.974 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026423134000000001 2023-10-30T06:31:09.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.974 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.974 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:16.875256+0000", 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0011054820000000001 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.976 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:82 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'52, mlcod=19'50)", 2023-10-30T06:31:09.976 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:09.976 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.67185090999999, 2023-10-30T06:31:09.976 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.028386602, 2023-10-30T06:31:09.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.977 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.977 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.977 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.978 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:09.978 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.978 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117309+0000", 2023-10-30T06:31:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.979 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117322+0000", 2023-10-30T06:31:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.309e-05 2023-10-30T06:31:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.980 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117545+0000", 2023-10-30T06:31:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00022343800000000001 2023-10-30T06:31:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.981 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117803+0000", 2023-10-30T06:31:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000258001 2023-10-30T06:31:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.982 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.117986+0000", 2023-10-30T06:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018322100000000001 2023-10-30T06:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.983 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119143+0000", 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001156645 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.984 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.119601+0000", 2023-10-30T06:31:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00045822799999999998 2023-10-30T06:31:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.985 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.142298+0000", 2023-10-30T06:31:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.022696555 2023-10-30T06:31:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.986 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145695+0000", 2023-10-30T06:31:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.003397424 2023-10-30T06:31:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:83 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'53, mlcod=19'50)", 2023-10-30T06:31:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.66435271799998, 2023-10-30T06:31:09.988 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041136530999999997, 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:09.989 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:09.990 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.990 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.990 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124807+0000", 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:09.991 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124816+0000", 2023-10-30T06:31:09.992 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.1970000000000002e-06 2023-10-30T06:31:09.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.992 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.992 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:09.992 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.124939+0000", 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00012329100000000001 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125121+0000", 2023-10-30T06:31:09.993 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018151300000000001 2023-10-30T06:31:09.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.994 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.994 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:09.994 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125231+0000", 2023-10-30T06:31:09.994 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010974199999999999 2023-10-30T06:31:09.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.995 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.995 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:09.995 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126215+0000", 2023-10-30T06:31:09.995 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00098434400000000006 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.126580+0000", 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000365101 2023-10-30T06:31:09.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.997 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.997 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:09.997 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165314+0000", 2023-10-30T06:31:09.997 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.038734032000000002 2023-10-30T06:31:09.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.998 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.998 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:09.998 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165943+0000", 2023-10-30T06:31:09.998 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00062931100000000002 2023-10-30T06:31:09.998 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.999 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:09.999 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:09.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:09.999 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:09.999 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:84 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'54, mlcod=19'50)", 2023-10-30T06:31:10.000 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:10.000 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.663691513, 2023-10-30T06:31:10.000 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.041063448000000002, 2023-10-30T06:31:10.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.001 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.001 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.001 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.001 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:10.001 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.002 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.002 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.002 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125468+0000", 2023-10-30T06:31:10.003 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.003 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.003 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.003 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.125474+0000", 2023-10-30T06:31:10.004 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.609e-06 2023-10-30T06:31:10.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.004 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.004 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.004 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.129920+0000", 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0044462199999999999 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130052+0000", 2023-10-30T06:31:10.005 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00013168399999999999 2023-10-30T06:31:10.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.006 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.006 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.006 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.130161+0000", 2023-10-30T06:31:10.006 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000109039 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132011+0000", 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001850197 2023-10-30T06:31:10.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.008 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.008 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.132380+0000", 2023-10-30T06:31:10.008 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036951099999999999 2023-10-30T06:31:10.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.009 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.009 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.009 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.165971+0000", 2023-10-30T06:31:10.009 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.033591154999999998 2023-10-30T06:31:10.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.010 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.010 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.010 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166532+0000", 2023-10-30T06:31:10.010 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00056003300000000004 2023-10-30T06:31:10.010 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:85 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'55, mlcod=19'50)", 2023-10-30T06:31:10.011 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:10.012 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.651105675, 2023-10-30T06:31:10.012 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.029195283999999998, 2023-10-30T06:31:10.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.012 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.014 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.014 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138054+0000", 2023-10-30T06:31:10.014 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.014 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.015 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.015 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138062+0000", 2023-10-30T06:31:10.015 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.4919999999999993e-06 2023-10-30T06:31:10.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138208+0000", 2023-10-30T06:31:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014587199999999999 2023-10-30T06:31:10.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.017 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.017 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.017 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138350+0000", 2023-10-30T06:31:10.017 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00014143900000000001 2023-10-30T06:31:10.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.018 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.018 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.138469+0000", 2023-10-30T06:31:10.018 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00011910599999999999 2023-10-30T06:31:10.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.019 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.019 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139337+0000", 2023-10-30T06:31:10.019 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00086850999999999996 2023-10-30T06:31:10.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.020 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.020 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.139702+0000", 2023-10-30T06:31:10.020 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00036454100000000002 2023-10-30T06:31:10.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.021 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.021 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.021 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.166719+0000", 2023-10-30T06:31:10.021 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.027017316999999999 2023-10-30T06:31:10.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.022 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.022 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.022 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.167249+0000", 2023-10-30T06:31:10.022 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00053000699999999998 2023-10-30T06:31:10.022 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.023 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.023 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.023 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.023 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:86 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'56, mlcod=19'50)", 2023-10-30T06:31:10.024 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:10.024 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.64501346200001, 2023-10-30T06:31:10.024 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.030627253, 2023-10-30T06:31:10.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.025 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.025 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.025 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.025 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:10.026 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.026 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.026 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144146+0000", 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144154+0000", 2023-10-30T06:31:10.027 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 8.0439999999999996e-06 2023-10-30T06:31:10.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.028 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.028 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144388+0000", 2023-10-30T06:31:10.028 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000234255 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144734+0000", 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00034573999999999998 2023-10-30T06:31:10.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.030 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.030 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.030 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.144859+0000", 2023-10-30T06:31:10.030 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000124955 2023-10-30T06:31:10.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.031 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.031 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.031 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148119+0000", 2023-10-30T06:31:10.031 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0032600440000000001 2023-10-30T06:31:10.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.032 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.033 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.033 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.148516+0000", 2023-10-30T06:31:10.034 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00039682999999999998 2023-10-30T06:31:10.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.034 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.034 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.034 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174082+0000", 2023-10-30T06:31:10.035 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.025565603999999999 2023-10-30T06:31:10.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.035 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.035 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.035 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.174773+0000", 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00069178100000000004 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.036 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.037 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:87 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'57, mlcod=19'50)", 2023-10-30T06:31:10.037 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:10.037 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 412.64401812099999, 2023-10-30T06:31:10.037 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031253535999999998, 2023-10-30T06:31:10.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.038 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.038 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.038 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.039 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145141+0000", 2023-10-30T06:31:10.040 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.040 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.040 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.145147+0000", 2023-10-30T06:31:10.041 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 5.7100000000000004e-06 2023-10-30T06:31:10.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.041 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.041 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.041 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.146471+0000", 2023-10-30T06:31:10.042 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0013241559999999999 2023-10-30T06:31:10.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.042 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.042 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.043 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147213+0000", 2023-10-30T06:31:10.043 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00074160000000000003 2023-10-30T06:31:10.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.043 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.043 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.147316+0000", 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00010343 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.044 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153553+0000", 2023-10-30T06:31:10.045 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0062362329999999999 2023-10-30T06:31:10.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.045 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.045 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.153907+0000", 2023-10-30T06:31:10.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00035421900000000002 2023-10-30T06:31:10.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.046 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.046 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.046 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.175734+0000", 2023-10-30T06:31:10.047 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.021827373000000001 2023-10-30T06:31:10.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.047 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.047 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:17.176395+0000", 2023-10-30T06:31:10.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066081500000000004 2023-10-30T06:31:10.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.048 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.049 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.049 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:104 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'66, mlcod=20'64)", 2023-10-30T06:31:10.049 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:10.049 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 405.32134554999999, 2023-10-30T06:31:10.049 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.026589738000000002, 2023-10-30T06:31:10.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.050 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.050 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.050 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:10.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467814+0000", 2023-10-30T06:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.053 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467824+0000", 2023-10-30T06:31:10.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.9869999999999994e-06 2023-10-30T06:31:10.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.467975+0000", 2023-10-30T06:31:10.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000151249 2023-10-30T06:31:10.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.054 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.054 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468155+0000", 2023-10-30T06:31:10.055 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000179631 2023-10-30T06:31:10.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.056 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.468281+0000", 2023-10-30T06:31:10.056 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000125611 2023-10-30T06:31:10.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.056 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.057 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.057 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469185+0000", 2023-10-30T06:31:10.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00090430600000000001 2023-10-30T06:31:10.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.469572+0000", 2023-10-30T06:31:10.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000387005 2023-10-30T06:31:10.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.493738+0000", 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.024166317 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.060 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.494404+0000", 2023-10-30T06:31:10.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00066563199999999999 2023-10-30T06:31:10.060 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.060 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.061 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.061 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.061 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_repop(client.4168.0:107 1.0 e24/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 24'69, mlcod=20'64)", 2023-10-30T06:31:10.061 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 405.28790406500002, 2023-10-30T06:31:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.031668847, 2023-10-30T06:31:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "commit sent; apply or cleanup", 2023-10-30T06:31:10.062 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:10.063 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.063 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:10.063 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:10.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.064 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:10.064 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501256+0000", 2023-10-30T06:31:10.064 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:10.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.065 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:10.065 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.501265+0000", 2023-10-30T06:31:10.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 9.0890000000000006e-06 2023-10-30T06:31:10.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.066 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.066 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:10.066 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510608+0000", 2023-10-30T06:31:10.066 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0093432020000000001 2023-10-30T06:31:10.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510791+0000", 2023-10-30T06:31:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018308300000000001 2023-10-30T06:31:10.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:10.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.510907+0000", 2023-10-30T06:31:10.068 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000115946 2023-10-30T06:31:10.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.069 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.069 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:10.069 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512147+0000", 2023-10-30T06:31:10.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0012399329999999999 2023-10-30T06:31:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "started", 2023-10-30T06:31:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.512592+0000", 2023-10-30T06:31:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00044530699999999998 2023-10-30T06:31:10.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "commit_sent", 2023-10-30T06:31:10.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.531826+0000", 2023-10-30T06:31:10.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.019234192000000001 2023-10-30T06:31:10.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:10.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:10.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:10.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:24:24.532924+0000", 2023-10-30T06:31:10.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0010980950000000001 2023-10-30T06:31:10.073 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.073 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.073 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.073 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:10.073 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:10.074 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-10-30T06:31:10.202 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:10.203 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:10.203 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:10.203 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-10-30T06:31:10.356 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:10.356 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:10.357 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:10.357 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:10.357 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-10-30T06:31:10.430 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:10.431 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "epoch": 125, 2023-10-30T06:31:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "fsid": "09a24e68-832d-4fbc-9604-48e808368d81", 2023-10-30T06:31:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "created": "2023-10-30T06:23:23.243533+0000", 2023-10-30T06:31:10.431 INFO:teuthology.orchestra.run.smithi079.stdout: "modified": "2023-10-30T06:31:07.066191+0000", 2023-10-30T06:31:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "last_up_change": "2023-10-30T06:24:12.541257+0000", 2023-10-30T06:31:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "last_in_change": "2023-10-30T06:30:59.454879+0000", 2023-10-30T06:31:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-30T06:31:10.432 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_num": 5804032, 2023-10-30T06:31:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_set": [ 2023-10-30T06:31:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "nodeep-scrub", 2023-10-30T06:31:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "pglog_hardlimit", 2023-10-30T06:31:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "purged_snapdirs", 2023-10-30T06:31:10.433 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_deletes", 2023-10-30T06:31:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "sortbitwise" 2023-10-30T06:31:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_version": 7, 2023-10-30T06:31:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "full_ratio": 0.94999998807907104, 2023-10-30T06:31:10.434 INFO:teuthology.orchestra.run.smithi079.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "nearfull_ratio": 0.85000002384185791, 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_snapshot": "", 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_max": 2, 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "max_osd": 8, 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "require_min_compat_client": "luminous", 2023-10-30T06:31:10.435 INFO:teuthology.orchestra.run.smithi079.stdout: "min_compat_client": "luminous", 2023-10-30T06:31:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "require_osd_release": "reef", 2023-10-30T06:31:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "allow_crimson": false, 2023-10-30T06:31:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pools": [ 2023-10-30T06:31:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.436 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 1, 2023-10-30T06:31:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": ".mgr", 2023-10-30T06:31:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:09.653526+0000", 2023-10-30T06:31:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:31:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:31:10.437 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:31:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:31:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:31:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:31:10.438 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:31:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:31:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:31:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:31:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:31:10.439 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:31:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 1, 2023-10-30T06:31:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 1, 2023-10-30T06:31:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 1, 2023-10-30T06:31:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 1, 2023-10-30T06:31:10.440 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 1, 2023-10-30T06:31:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:31:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:31:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:31:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:31:10.441 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:31:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:31:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:31:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "20", 2023-10-30T06:31:10.442 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:31:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:31:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:31:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:31:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:31:10.443 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:31:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:31:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:31:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:31:10.444 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:31:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:31:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:31:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:31:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:31:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:31:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:31:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:31:10.446 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:31:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:31:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:31:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:31:10.447 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:31:10.448 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:31:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:31:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:31:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:31:10.449 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:31:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:31:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:31:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:31:10.450 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:31:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:31:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:31:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:31:10.451 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:31:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:31:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:31:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "options": { 2023-10-30T06:31:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_max": 32, 2023-10-30T06:31:10.452 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_min": 1 2023-10-30T06:31:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:31:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": {} 2023-10-30T06:31:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.453 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:31:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 6.9000000953674316, 2023-10-30T06:31:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 6.9000000953674316, 2023-10-30T06:31:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.28999999165534973, 2023-10-30T06:31:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 2, 2023-10-30T06:31:10.454 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 2, 2023-10-30T06:31:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.69626235961914062, 2023-10-30T06:31:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.80000001192092896, 2023-10-30T06:31:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.69999998807907104 2023-10-30T06:31:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.455 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "pool": 2, 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_name": "rbd", 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "create_time": "2023-10-30T06:24:17.127645+0000", 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flags": 1, 2023-10-30T06:31:10.456 INFO:teuthology.orchestra.run.smithi079.stdout: "flags_names": "hashpspool", 2023-10-30T06:31:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "type": 1, 2023-10-30T06:31:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "size": 2, 2023-10-30T06:31:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "min_size": 1, 2023-10-30T06:31:10.457 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_rule": 0, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_count": 0, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_target": 0, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_barrier": 0, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "object_hash": 2, 2023-10-30T06:31:10.458 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_autoscale_mode": "off", 2023-10-30T06:31:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num": 8, 2023-10-30T06:31:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num": 8, 2023-10-30T06:31:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_placement_num_target": 8, 2023-10-30T06:31:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_target": 8, 2023-10-30T06:31:10.459 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_num_pending": 8, 2023-10-30T06:31:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "last_pg_merge_meta": { 2023-10-30T06:31:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "source_pgid": "0.0", 2023-10-30T06:31:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "ready_epoch": 0, 2023-10-30T06:31:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_started": 0, 2023-10-30T06:31:10.460 INFO:teuthology.orchestra.run.smithi079.stdout: "last_epoch_clean": 0, 2023-10-30T06:31:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "source_version": "0'0", 2023-10-30T06:31:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "target_version": "0'0" 2023-10-30T06:31:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "last_change": "24", 2023-10-30T06:31:10.461 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend": "0", 2023-10-30T06:31:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-30T06:31:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "last_force_op_resend_preluminous": "0", 2023-10-30T06:31:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "auid": 0, 2023-10-30T06:31:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_mode": "selfmanaged", 2023-10-30T06:31:10.462 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_seq": 0, 2023-10-30T06:31:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "snap_epoch": 0, 2023-10-30T06:31:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "pool_snaps": [], 2023-10-30T06:31:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps": "[]", 2023-10-30T06:31:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_bytes": 0, 2023-10-30T06:31:10.463 INFO:teuthology.orchestra.run.smithi079.stdout: "quota_max_objects": 0, 2023-10-30T06:31:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tiers": [], 2023-10-30T06:31:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "tier_of": -1, 2023-10-30T06:31:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "read_tier": -1, 2023-10-30T06:31:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "write_tier": -1, 2023-10-30T06:31:10.464 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_mode": "none", 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_bytes": 0, 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "target_max_objects": 0, 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-30T06:31:10.465 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_flush_age": 0, 2023-10-30T06:31:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "cache_min_evict_age": 0, 2023-10-30T06:31:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profile": "", 2023-10-30T06:31:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_params": { 2023-10-30T06:31:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "none" 2023-10-30T06:31:10.466 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_period": 0, 2023-10-30T06:31:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_count": 0, 2023-10-30T06:31:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "use_gmt_hitset": true, 2023-10-30T06:31:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "min_read_recency_for_promote": 0, 2023-10-30T06:31:10.467 INFO:teuthology.orchestra.run.smithi079.stdout: "min_write_recency_for_promote": 0, 2023-10-30T06:31:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_grade_decay_rate": 0, 2023-10-30T06:31:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "hit_set_search_last_n": 0, 2023-10-30T06:31:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "grade_table": [], 2023-10-30T06:31:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "stripe_width": 0, 2023-10-30T06:31:10.468 INFO:teuthology.orchestra.run.smithi079.stdout: "expected_num_objects": 0, 2023-10-30T06:31:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "fast_read": false, 2023-10-30T06:31:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "options": {}, 2023-10-30T06:31:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "application_metadata": { 2023-10-30T06:31:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: "rbd": {} 2023-10-30T06:31:10.469 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "read_balance": { 2023-10-30T06:31:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "score_acting": 4.3600001335144043, 2023-10-30T06:31:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "score_stable": 4.3600001335144043, 2023-10-30T06:31:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "optimal_score": 0.86000001430511475, 2023-10-30T06:31:10.470 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_acting": 3.75, 2023-10-30T06:31:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "raw_score_stable": 3.75, 2023-10-30T06:31:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity_weighted": 0.69626235961914062, 2023-10-30T06:31:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity": 0.80000001192092896, 2023-10-30T06:31:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: "average_primary_affinity_weighted": 0.69999998807907104 2023-10-30T06:31:10.471 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:31:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.472 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:31:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6e79cc30-6aa7-489d-8806-48c107644b8b", 2023-10-30T06:31:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 0, 2023-10-30T06:31:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 0, 2023-10-30T06:31:10.473 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:31:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:31:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:31:10.474 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.475 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6810", 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.476 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6811", 2023-10-30T06:31:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.477 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6812", 2023-10-30T06:31:10.478 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6813", 2023-10-30T06:31:10.479 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.480 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6816", 2023-10-30T06:31:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.481 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6817", 2023-10-30T06:31:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.482 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.483 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6814", 2023-10-30T06:31:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.484 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6815", 2023-10-30T06:31:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2677821092 2023-10-30T06:31:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.485 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6811/2677821092", 2023-10-30T06:31:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6813/2677821092", 2023-10-30T06:31:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6817/2677821092", 2023-10-30T06:31:10.486 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6815/2677821092", 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.487 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:31:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "c8a16891-3cdc-487f-9fdc-1b88ce43caf4", 2023-10-30T06:31:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.488 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:31:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.489 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 18, 2023-10-30T06:31:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:31:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.490 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6826", 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.491 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6827", 2023-10-30T06:31:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.492 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.493 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6828", 2023-10-30T06:31:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.494 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6829", 2023-10-30T06:31:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.495 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.496 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6832", 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.497 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6833", 2023-10-30T06:31:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.498 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6830", 2023-10-30T06:31:10.499 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6831", 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2076704984 2023-10-30T06:31:10.500 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6827/2076704984", 2023-10-30T06:31:10.501 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6829/2076704984", 2023-10-30T06:31:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6833/2076704984", 2023-10-30T06:31:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6831/2076704984", 2023-10-30T06:31:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.502 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "0b1c9a57-cced-4163-a178-f461fc9596aa", 2023-10-30T06:31:10.503 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0, 2023-10-30T06:31:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.504 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:31:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 94, 2023-10-30T06:31:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.505 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6818", 2023-10-30T06:31:10.506 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6819", 2023-10-30T06:31:10.507 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.508 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6820", 2023-10-30T06:31:10.509 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.510 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6821", 2023-10-30T06:31:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.511 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6824", 2023-10-30T06:31:10.512 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6825", 2023-10-30T06:31:10.513 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.514 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6822", 2023-10-30T06:31:10.515 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6823", 2023-10-30T06:31:10.516 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3346893925 2023-10-30T06:31:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6819/3346893925", 2023-10-30T06:31:10.517 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6821/3346893925", 2023-10-30T06:31:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6825/3346893925", 2023-10-30T06:31:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6823/3346893925", 2023-10-30T06:31:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.518 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:31:10.519 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "eab98798-a4c8-4e5f-83ec-40b3f31a4d50", 2023-10-30T06:31:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:31:10.520 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:31:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 122, 2023-10-30T06:31:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.521 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.522 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6802", 2023-10-30T06:31:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.523 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6803", 2023-10-30T06:31:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.524 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6804", 2023-10-30T06:31:10.525 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6805", 2023-10-30T06:31:10.526 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.527 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6808", 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.528 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6809", 2023-10-30T06:31:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.529 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.530 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6806", 2023-10-30T06:31:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.531 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.79:6807", 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1649947840 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.532 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.79:6803/1649947840", 2023-10-30T06:31:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.79:6805/1649947840", 2023-10-30T06:31:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.79:6809/1649947840", 2023-10-30T06:31:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.79:6807/1649947840", 2023-10-30T06:31:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.533 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.534 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:31:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "6f17fdbb-5762-457a-8b89-4c5a4f2197b2", 2023-10-30T06:31:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.535 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:31:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:31:10.536 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 122, 2023-10-30T06:31:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.537 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.538 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.538 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6824", 2023-10-30T06:31:10.538 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.538 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.538 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6825", 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.539 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.540 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.540 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.540 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.540 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.540 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6826", 2023-10-30T06:31:10.541 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.541 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.541 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.541 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.541 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6827", 2023-10-30T06:31:10.542 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.542 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.542 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.542 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.542 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.543 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.543 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.543 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.543 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6830", 2023-10-30T06:31:10.543 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6831", 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.544 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.545 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.545 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.545 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.545 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.545 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.546 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.546 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6828", 2023-10-30T06:31:10.546 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.546 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.546 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.547 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.547 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6829", 2023-10-30T06:31:10.547 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2788056220 2023-10-30T06:31:10.547 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.547 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6825/2788056220", 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6827/2788056220", 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6831/2788056220", 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6829/2788056220", 2023-10-30T06:31:10.548 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.549 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.549 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.549 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.549 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.549 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.550 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:31:10.550 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "49ea8aae-4b05-43c9-979a-99026ae58a89", 2023-10-30T06:31:10.550 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.550 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.550 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 1, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 69, 2023-10-30T06:31:10.551 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.552 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.552 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.552 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.552 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.552 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6808", 2023-10-30T06:31:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.553 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.554 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6809", 2023-10-30T06:31:10.554 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.554 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.554 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.554 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.555 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.555 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.555 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.555 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.555 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6810", 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6811", 2023-10-30T06:31:10.556 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.557 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.558 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.558 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6814", 2023-10-30T06:31:10.558 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.558 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.558 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.559 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.559 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6815", 2023-10-30T06:31:10.559 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.559 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.560 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.561 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6812", 2023-10-30T06:31:10.561 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.561 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.561 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.561 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.562 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6813", 2023-10-30T06:31:10.562 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2352822260 2023-10-30T06:31:10.562 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.562 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.562 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6809/2352822260", 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6811/2352822260", 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6815/2352822260", 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6813/2352822260", 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.563 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:31:10.564 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "ebb325d3-e1fc-44b6-929b-b7fdd71f82b1", 2023-10-30T06:31:10.565 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.565 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.565 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.565 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.22564697265625, 2023-10-30T06:31:10.565 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.566 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.566 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 16, 2023-10-30T06:31:10.566 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 0, 2023-10-30T06:31:10.566 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.567 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.567 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.567 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.567 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.567 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.568 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6800", 2023-10-30T06:31:10.568 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.568 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.568 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.569 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.569 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6801", 2023-10-30T06:31:10.569 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.569 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.569 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.570 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.570 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.570 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.570 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.570 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6802", 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.571 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6803", 2023-10-30T06:31:10.572 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.572 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.572 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.572 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.573 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.573 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.573 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.573 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.573 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6806", 2023-10-30T06:31:10.574 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.574 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.574 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.574 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.574 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6807", 2023-10-30T06:31:10.575 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.575 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.575 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.575 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.575 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.576 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.576 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.576 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.576 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6804", 2023-10-30T06:31:10.576 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6805", 2023-10-30T06:31:10.577 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 4229386639 2023-10-30T06:31:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6801/4229386639", 2023-10-30T06:31:10.578 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6803/4229386639", 2023-10-30T06:31:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6807/4229386639", 2023-10-30T06:31:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6805/4229386639", 2023-10-30T06:31:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.579 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "uuid": "31813e1b-e740-49b2-9173-85911e32f3d9", 2023-10-30T06:31:10.580 INFO:teuthology.orchestra.run.smithi079.stdout: "up": 1, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "in": 1, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "weight": 1, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_affinity": 0.344451904296875, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_begin": 0, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "last_clean_end": 0, 2023-10-30T06:31:10.581 INFO:teuthology.orchestra.run.smithi079.stdout: "up_from": 15, 2023-10-30T06:31:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "up_thru": 114, 2023-10-30T06:31:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "down_at": 0, 2023-10-30T06:31:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "lost_at": 0, 2023-10-30T06:31:10.582 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addrs": { 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6816", 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6817", 2023-10-30T06:31:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addrs": { 2023-10-30T06:31:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.585 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6818", 2023-10-30T06:31:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.586 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6819", 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.587 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addrs": { 2023-10-30T06:31:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.588 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6822", 2023-10-30T06:31:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.589 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6823", 2023-10-30T06:31:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.590 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addrs": { 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "addrvec": [ 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v2", 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6820", 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-10-30T06:31:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "172.21.15.112:6821", 2023-10-30T06:31:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 3711700983 2023-10-30T06:31:10.592 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "public_addr": "172.21.15.112:6817/3711700983", 2023-10-30T06:31:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_addr": "172.21.15.112:6819/3711700983", 2023-10-30T06:31:10.593 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_back_addr": "172.21.15.112:6823/3711700983", 2023-10-30T06:31:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "heartbeat_front_addr": "172.21.15.112:6821/3711700983", 2023-10-30T06:31:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "state": [ 2023-10-30T06:31:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "exists", 2023-10-30T06:31:10.594 INFO:teuthology.orchestra.run.smithi079.stdout: "up" 2023-10-30T06:31:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.595 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_xinfo": [ 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 0, 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.596 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 65536, 2023-10-30T06:31:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.906453+0000", 2023-10-30T06:31:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 1, 2023-10-30T06:31:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.598 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:10.017307+0000", 2023-10-30T06:31:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.600 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.600 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 2, 2023-10-30T06:31:10.600 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.600 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.600 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.601 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.601 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.601 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:08.611770+0000", 2023-10-30T06:31:10.601 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.602 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.602 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 3, 2023-10-30T06:31:10.602 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.602 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.602 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.603 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.603 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.603 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.200083+0000", 2023-10-30T06:31:10.603 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 4, 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.604 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.605 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.605 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.200292+0000", 2023-10-30T06:31:10.605 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.605 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.605 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.606 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 5, 2023-10-30T06:31:10.606 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.606 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.606 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.607 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.607 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.607 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.555977+0000", 2023-10-30T06:31:10.607 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.607 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.608 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.608 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 6, 2023-10-30T06:31:10.608 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.608 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.608 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.609 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.609 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.609 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:07.001381+0000", 2023-10-30T06:31:10.609 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.610 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.610 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": 7, 2023-10-30T06:31:10.610 INFO:teuthology.orchestra.run.smithi079.stdout: "down_stamp": "0.000000", 2023-10-30T06:31:10.610 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_probability": 0, 2023-10-30T06:31:10.611 INFO:teuthology.orchestra.run.smithi079.stdout: "laggy_interval": 0, 2023-10-30T06:31:10.611 INFO:teuthology.orchestra.run.smithi079.stdout: "features": 4540138322906710015, 2023-10-30T06:31:10.611 INFO:teuthology.orchestra.run.smithi079.stdout: "old_weight": 0, 2023-10-30T06:31:10.611 INFO:teuthology.orchestra.run.smithi079.stdout: "last_purged_snaps_scrub": "2023-10-30T06:24:06.548682+0000", 2023-10-30T06:31:10.612 INFO:teuthology.orchestra.run.smithi079.stdout: "dead_epoch": 0 2023-10-30T06:31:10.612 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.612 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.612 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap": [ 2023-10-30T06:31:10.612 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.613 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.5", 2023-10-30T06:31:10.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osds": [ 2023-10-30T06:31:10.613 INFO:teuthology.orchestra.run.smithi079.stdout: 7, 2023-10-30T06:31:10.613 INFO:teuthology.orchestra.run.smithi079.stdout: 0 2023-10-30T06:31:10.613 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.614 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.614 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_items": [ 2023-10-30T06:31:10.614 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pgid": "2.0", 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: "mappings": [ 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: "from": 5, 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: "to": 2 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.615 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-10-30T06:31:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: ], 2023-10-30T06:31:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_upmap_primaries": [], 2023-10-30T06:31:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: "pg_temp": [], 2023-10-30T06:31:10.616 INFO:teuthology.orchestra.run.smithi079.stdout: "primary_temp": [], 2023-10-30T06:31:10.617 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklist": {}, 2023-10-30T06:31:10.617 INFO:teuthology.orchestra.run.smithi079.stdout: "range_blocklist": {}, 2023-10-30T06:31:10.617 INFO:teuthology.orchestra.run.smithi079.stdout: "erasure_code_profiles": { 2023-10-30T06:31:10.617 INFO:teuthology.orchestra.run.smithi079.stdout: "default": { 2023-10-30T06:31:10.617 INFO:teuthology.orchestra.run.smithi079.stdout: "crush-failure-domain": "osd", 2023-10-30T06:31:10.618 INFO:teuthology.orchestra.run.smithi079.stdout: "k": "2", 2023-10-30T06:31:10.618 INFO:teuthology.orchestra.run.smithi079.stdout: "m": "1", 2023-10-30T06:31:10.618 INFO:teuthology.orchestra.run.smithi079.stdout: "plugin": "jerasure", 2023-10-30T06:31:10.618 INFO:teuthology.orchestra.run.smithi079.stdout: "technique": "reed_sol_van" 2023-10-30T06:31:10.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-10-30T06:31:10.619 INFO:teuthology.orchestra.run.smithi079.stdout: "removed_snaps_queue": [], 2023-10-30T06:31:10.619 INFO:teuthology.orchestra.run.smithi079.stdout: "new_removed_snaps": [], 2023-10-30T06:31:10.620 INFO:teuthology.orchestra.run.smithi079.stdout: "new_purged_snaps": [], 2023-10-30T06:31:10.620 INFO:teuthology.orchestra.run.smithi079.stdout: "crush_node_flags": {}, 2023-10-30T06:31:10.620 INFO:teuthology.orchestra.run.smithi079.stdout: "device_class_flags": {}, 2023-10-30T06:31:10.620 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode": { 2023-10-30T06:31:10.620 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_enabled": false, 2023-10-30T06:31:10.621 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_bucket_count": 0, 2023-10-30T06:31:10.621 INFO:teuthology.orchestra.run.smithi079.stdout: "degraded_stretch_mode": 0, 2023-10-30T06:31:10.621 INFO:teuthology.orchestra.run.smithi079.stdout: "recovering_stretch_mode": 0, 2023-10-30T06:31:10.621 INFO:teuthology.orchestra.run.smithi079.stdout: "stretch_mode_bucket": 0 2023-10-30T06:31:10.622 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-10-30T06:31:10.622 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:10.624 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:10.624 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [] 2023-10-30T06:31:10.624 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.625 INFO:tasks.thrashosds.thrasher:j is {'epoch': 125, 'fsid': '09a24e68-832d-4fbc-9604-48e808368d81', 'created': '2023-10-30T06:23:23.243533+0000', 'modified': '2023-10-30T06:31:07.066191+0000', 'last_up_change': '2023-10-30T06:24:12.541257+0000', 'last_in_change': '2023-10-30T06:30:59.454879+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-30T06:24:09.653526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6962623596191406, 'average_primary_affinity': 0.800000011920929, 'average_primary_affinity_weighted': 0.699999988079071}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-30T06:24:17.127645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.360000133514404, 'score_stable': 4.360000133514404, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.6962623596191406, 'average_primary_affinity': 0.800000011920929, 'average_primary_affinity_weighted': 0.699999988079071}}], 'osds': [{'osd': 0, 'uuid': '6e79cc30-6aa7-489d-8806-48c107644b8b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6810', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6811', 'nonce': 2677821092}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6812', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6813', 'nonce': 2677821092}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6816', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6817', 'nonce': 2677821092}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6814', 'nonce': 2677821092}, {'type': 'v1', 'addr': '172.21.15.79:6815', 'nonce': 2677821092}]}, 'public_addr': '172.21.15.79:6811/2677821092', 'cluster_addr': '172.21.15.79:6813/2677821092', 'heartbeat_back_addr': '172.21.15.79:6817/2677821092', 'heartbeat_front_addr': '172.21.15.79:6815/2677821092', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c8a16891-3cdc-487f-9fdc-1b88ce43caf4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6826', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6827', 'nonce': 2076704984}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6828', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6829', 'nonce': 2076704984}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2076704984}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6830', 'nonce': 2076704984}, {'type': 'v1', 'addr': '172.21.15.79:6831', 'nonce': 2076704984}]}, 'public_addr': '172.21.15.79:6827/2076704984', 'cluster_addr': '172.21.15.79:6829/2076704984', 'heartbeat_back_addr': '172.21.15.79:6833/2076704984', 'heartbeat_front_addr': '172.21.15.79:6831/2076704984', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0b1c9a57-cced-4163-a178-f461fc9596aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6818', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6819', 'nonce': 3346893925}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6820', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6821', 'nonce': 3346893925}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6824', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6825', 'nonce': 3346893925}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6822', 'nonce': 3346893925}, {'type': 'v1', 'addr': '172.21.15.79:6823', 'nonce': 3346893925}]}, 'public_addr': '172.21.15.79:6819/3346893925', 'cluster_addr': '172.21.15.79:6821/3346893925', 'heartbeat_back_addr': '172.21.15.79:6825/3346893925', 'heartbeat_front_addr': '172.21.15.79:6823/3346893925', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eab98798-a4c8-4e5f-83ec-40b3f31a4d50', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6802', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6803', 'nonce': 1649947840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6804', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6805', 'nonce': 1649947840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6808', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6809', 'nonce': 1649947840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6806', 'nonce': 1649947840}, {'type': 'v1', 'addr': '172.21.15.79:6807', 'nonce': 1649947840}]}, 'public_addr': '172.21.15.79:6803/1649947840', 'cluster_addr': '172.21.15.79:6805/1649947840', 'heartbeat_back_addr': '172.21.15.79:6809/1649947840', 'heartbeat_front_addr': '172.21.15.79:6807/1649947840', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6f17fdbb-5762-457a-8b89-4c5a4f2197b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 122, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6824', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6825', 'nonce': 2788056220}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6826', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6827', 'nonce': 2788056220}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6830', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6831', 'nonce': 2788056220}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6828', 'nonce': 2788056220}, {'type': 'v1', 'addr': '172.21.15.112:6829', 'nonce': 2788056220}]}, 'public_addr': '172.21.15.112:6825/2788056220', 'cluster_addr': '172.21.15.112:6827/2788056220', 'heartbeat_back_addr': '172.21.15.112:6831/2788056220', 'heartbeat_front_addr': '172.21.15.112:6829/2788056220', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '49ea8aae-4b05-43c9-979a-99026ae58a89', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6808', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6809', 'nonce': 2352822260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6810', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6811', 'nonce': 2352822260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6814', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6815', 'nonce': 2352822260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6812', 'nonce': 2352822260}, {'type': 'v1', 'addr': '172.21.15.112:6813', 'nonce': 2352822260}]}, 'public_addr': '172.21.15.112:6809/2352822260', 'cluster_addr': '172.21.15.112:6811/2352822260', 'heartbeat_back_addr': '172.21.15.112:6815/2352822260', 'heartbeat_front_addr': '172.21.15.112:6813/2352822260', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ebb325d3-e1fc-44b6-929b-b7fdd71f82b1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.22564697265625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6800', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6801', 'nonce': 4229386639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6802', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6803', 'nonce': 4229386639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6806', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6807', 'nonce': 4229386639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6804', 'nonce': 4229386639}, {'type': 'v1', 'addr': '172.21.15.112:6805', 'nonce': 4229386639}]}, 'public_addr': '172.21.15.112:6801/4229386639', 'cluster_addr': '172.21.15.112:6803/4229386639', 'heartbeat_back_addr': '172.21.15.112:6807/4229386639', 'heartbeat_front_addr': '172.21.15.112:6805/4229386639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '31813e1b-e740-49b2-9173-85911e32f3d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.344451904296875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 114, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6816', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6817', 'nonce': 3711700983}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6818', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6819', 'nonce': 3711700983}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6822', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6823', 'nonce': 3711700983}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6820', 'nonce': 3711700983}, {'type': 'v1', 'addr': '172.21.15.112:6821', 'nonce': 3711700983}]}, 'public_addr': '172.21.15.112:6817/3711700983', 'cluster_addr': '172.21.15.112:6819/3711700983', 'heartbeat_back_addr': '172.21.15.112:6823/3711700983', 'heartbeat_front_addr': '172.21.15.112:6821/3711700983', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.906453+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:10.017307+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:08.611770+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.200083+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.200292+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.555977+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:07.001381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-10-30T06:24:06.548682+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [7, 0]}], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-10-30T06:31:10.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:10.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-10-30T06:31:10.628 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:31:10.753 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:10.753 INFO:teuthology.orchestra.run.smithi112.stdout: "num_ops": 0 2023-10-30T06:31:10.754 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.762 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-10-30T06:31:10.905 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [], 2023-10-30T06:31:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "complaint_time": 30, 2023-10-30T06:31:10.906 INFO:teuthology.orchestra.run.smithi112.stdout: "num_blocked_ops": 0 2023-10-30T06:31:10.906 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:10.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-10-30T06:31:11.046 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-10-30T06:31:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "size": 20, 2023-10-30T06:31:11.046 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 600, 2023-10-30T06:31:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "ops": [ 2023-10-30T06:31:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2023-10-30T06:31:11.047 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 334.07165696099997, 2023-10-30T06:31:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0053161959999999996, 2023-10-30T06:31:11.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:31:11.049 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 4 2023-10-30T06:31:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.050 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:11.051 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:11.052 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969753+0000", 2023-10-30T06:31:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.053 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969776+0000", 2023-10-30T06:31:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 2.2751e-05 2023-10-30T06:31:11.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:11.055 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.969959+0000", 2023-10-30T06:31:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00018378099999999999 2023-10-30T06:31:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.057 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.970178+0000", 2023-10-30T06:31:11.058 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00021909300000000001 2023-10-30T06:31:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:11.059 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.972651+0000", 2023-10-30T06:31:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0024721600000000002 2023-10-30T06:31:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.060 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.973576+0000", 2023-10-30T06:31:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00092505899999999999 2023-10-30T06:31:11.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:11.062 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:25:36.975069+0000", 2023-10-30T06:31:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0014933520000000001 2023-10-30T06:31:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:11.063 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.064 INFO:teuthology.orchestra.run.smithi112.stdout: "description": "osd_op(client.4176.0:10 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2023-10-30T06:31:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "initiated_at": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "age": 154.051164637, 2023-10-30T06:31:11.065 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.0027355909999999999, 2023-10-30T06:31:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type_data": { 2023-10-30T06:31:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flag_point": "reached pg", 2023-10-30T06:31:11.066 INFO:teuthology.orchestra.run.smithi112.stdout: "client_info": { 2023-10-30T06:31:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "client": "client.4176", 2023-10-30T06:31:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "client_addr": "172.21.15.79:0/2560044989", 2023-10-30T06:31:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tid": 10 2023-10-30T06:31:11.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "events": [ 2023-10-30T06:31:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "initiated", 2023-10-30T06:31:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:11.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "header_read", 2023-10-30T06:31:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990245+0000", 2023-10-30T06:31:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0 2023-10-30T06:31:11.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "throttled", 2023-10-30T06:31:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990259+0000", 2023-10-30T06:31:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 1.3847e-05 2023-10-30T06:31:11.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "all_read", 2023-10-30T06:31:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990403+0000", 2023-10-30T06:31:11.072 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000144456 2023-10-30T06:31:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "dispatched", 2023-10-30T06:31:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990590+0000", 2023-10-30T06:31:11.073 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.000186734 2023-10-30T06:31:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "queued_for_pg", 2023-10-30T06:31:11.074 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.990748+0000", 2023-10-30T06:31:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00015797700000000001 2023-10-30T06:31:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "reached_pg", 2023-10-30T06:31:11.075 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.991638+0000", 2023-10-30T06:31:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.00088934799999999998 2023-10-30T06:31:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-10-30T06:31:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2023-10-30T06:31:11.076 INFO:teuthology.orchestra.run.smithi112.stdout: "event": "done", 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "time": "2023-10-30T06:28:36.992981+0000", 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: "duration": 0.001343229 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:11.077 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-10-30T06:31:11.078 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2023-10-30T06:31:11.078 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-10-30T06:31:11.241 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:11.242 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:11.253 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-10-30T06:31:10.599174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":194236,"kb_used_data":5096,"kb_used_omap":10,"kb_used_meta":189045,"kb_avail":660408644,"statfs":{"total":676457349120,"available":676258451456,"internally_reserved":0,"allocated":5218304,"data_stored":3100380,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":11150,"internal_metadata":193582194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.273695"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:01.171613+0000","last_change":"2023-10-30T06:31:01.163727+0000","last_active":"2023-10-30T06:31:01.171613+0000","last_peered":"2023-10-30T06:31:01.171613+0000","last_clean":"2023-10-30T06:31:01.171613+0000","last_became_active":"2023-10-30T06:31:01.157700+0000","last_became_peered":"2023-10-30T06:31:01.157700+0000","last_unstale":"2023-10-30T06:31:01.171613+0000","last_undegraded":"2023-10-30T06:31:01.171613+0000","last_fullsized":"2023-10-30T06:31:01.171613+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:44:37.922369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":119,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:00.847607+0000","last_change":"2023-10-30T06:31:00.839873+0000","last_active":"2023-10-30T06:31:00.847607+0000","last_peered":"2023-10-30T06:31:00.847607+0000","last_clean":"2023-10-30T06:31:00.847607+0000","last_became_active":"2023-10-30T06:31:00.833858+0000","last_became_peered":"2023-10-30T06:31:00.833858+0000","last_unstale":"2023-10-30T06:31:00.847607+0000","last_undegraded":"2023-10-30T06:31:00.847607+0000","last_fullsized":"2023-10-30T06:31:00.847607+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:11:50.977375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:02.081597+0000","last_change":"2023-10-30T06:31:00.753665+0000","last_active":"2023-10-30T06:31:02.081597+0000","last_peered":"2023-10-30T06:31:02.081597+0000","last_clean":"2023-10-30T06:31:02.081597+0000","last_became_active":"2023-10-30T06:31:00.747430+0000","last_became_peered":"2023-10-30T06:31:00.747430+0000","last_unstale":"2023-10-30T06:31:02.081597+0000","last_undegraded":"2023-10-30T06:31:02.081597+0000","last_fullsized":"2023-10-30T06:31:02.081597+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:55:51.946102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":128,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:00.824898+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:00.824898+0000","last_peered":"2023-10-30T06:31:00.824898+0000","last_clean":"2023-10-30T06:31:00.824898+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:00.824898+0000","last_undegraded":"2023-10-30T06:31:00.824898+0000","last_fullsized":"2023-10-30T06:31:00.824898+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:00.709276+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:00.709276+0000","last_peered":"2023-10-30T06:31:00.709276+0000","last_clean":"2023-10-30T06:31:00.709276+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:00.709276+0000","last_undegraded":"2023-10-30T06:31:00.709276+0000","last_fullsized":"2023-10-30T06:31:00.709276+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:01.122355+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:01.122355+0000","last_peered":"2023-10-30T06:31:01.122355+0000","last_clean":"2023-10-30T06:31:01.122355+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:01.122355+0000","last_undegraded":"2023-10-30T06:31:01.122355+0000","last_fullsized":"2023-10-30T06:31:01.122355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:00.711172+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:00.711172+0000","last_peered":"2023-10-30T06:31:00.711172+0000","last_clean":"2023-10-30T06:31:00.711172+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:00.711172+0000","last_undegraded":"2023-10-30T06:31:00.711172+0000","last_fullsized":"2023-10-30T06:31:00.711172+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-30T06:31:00.724383+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:00.724383+0000","last_peered":"2023-10-30T06:31:00.724383+0000","last_clean":"2023-10-30T06:31:00.724383+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:00.724383+0000","last_undegraded":"2023-10-30T06:31:00.724383+0000","last_fullsized":"2023-10-30T06:31:00.724383+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":218,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-30T06:31:04.625812+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:04.625812+0000","last_peered":"2023-10-30T06:31:04.625812+0000","last_clean":"2023-10-30T06:31:04.625812+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:04.625812+0000","last_undegraded":"2023-10-30T06:31:04.625812+0000","last_fullsized":"2023-10-30T06:31:04.625812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411415,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731822,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":11.656000000000001,"5min":10.69,"15min":10.470000000000001},"min":{"1min":4.6680000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":41.893999999999998,"5min":51.225000000000001,"15min":51.225000000000001},"last":5.5869999999999997},{"interface":"front","average":{"1min":9.5790000000000006,"5min":10.984,"15min":11.715999999999999},"min":{"1min":5.1260000000000003,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":19.98,"5min":54.011000000000003,"15min":54.011000000000003},"last":7.21}]},{"osd":2,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":11.449,"5min":11.566000000000001,"15min":13.054},"min":{"1min":6.8380000000000001,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":19.623999999999999,"5min":44.537999999999997,"15min":52.552},"last":9.0619999999999994},{"interface":"front","average":{"1min":15.808,"5min":14.797000000000001,"15min":15.029999999999999},"min":{"1min":8.6440000000000001,"5min":7.391,"15min":7.1589999999999998},"max":{"1min":40.183,"5min":55.585999999999999,"15min":55.585999999999999},"last":11.313000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":22.440999999999999,"5min":16.683,"15min":21.021000000000001},"min":{"1min":10.663,"5min":8.5980000000000008,"15min":8.5980000000000008},"max":{"1min":48.716000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":12.106999999999999},{"interface":"front","average":{"1min":17.629000000000001,"5min":14.619,"15min":19.920999999999999},"min":{"1min":8.3030000000000008,"5min":7.1849999999999996,"15min":7.1849999999999996},"max":{"1min":50.332999999999998,"5min":50.332999999999998,"15min":124.075},"last":13.654}]},{"osd":4,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":17.442,"5min":13.93,"15min":13.76},"min":{"1min":8.2159999999999993,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":48.134999999999998,"5min":48.134999999999998,"15min":51.649000000000001},"last":14.295},{"interface":"front","average":{"1min":18.248000000000001,"5min":17.190000000000001,"15min":17.728999999999999},"min":{"1min":9.1440000000000001,"5min":8.8810000000000002,"15min":7.9400000000000004},"max":{"1min":46.308,"5min":46.308,"15min":46.308},"last":9.9309999999999992}]},{"osd":5,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":16.280999999999999,"5min":16.262,"15min":19.454999999999998},"min":{"1min":8.0500000000000007,"5min":7.351,"15min":7.351},"max":{"1min":26.207999999999998,"5min":58.648000000000003,"15min":123.754},"last":12.792999999999999},{"interface":"front","average":{"1min":17.007000000000001,"5min":15.577999999999999,"15min":20.579000000000001},"min":{"1min":6.8520000000000003,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":31.82,"5min":41.558999999999997,"15min":122.72499999999999},"last":20.675000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":17.536999999999999,"15min":20.149999999999999},"min":{"1min":9.8979999999999997,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":27.460999999999999,"5min":39.860999999999997,"15min":66.400999999999996},"last":18.131},{"interface":"front","average":{"1min":19.936,"5min":17.399999999999999,"15min":19.039000000000001},"min":{"1min":11.220000000000001,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":32.485999999999997,"5min":43.159999999999997,"15min":53.149000000000001},"last":15.882}]},{"osd":7,"last update":"Mon Oct 30 06:30:27 2023","interfaces":[{"interface":"back","average":{"1min":22.295999999999999,"5min":19.056000000000001,"15min":20.693999999999999},"min":{"1min":11.384,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":43.174999999999997,"5min":51.375,"15min":51.375},"last":14.9},{"interface":"front","average":{"1min":19.138999999999999,"5min":19.327000000000002,"15min":20.838999999999999},"min":{"1min":8.6739999999999995,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":29.103999999999999,"5min":54.439,"15min":69.991},"last":19.34}]}]},{"osd":6,"up_from":16,"seq":68719476824,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":10.955,"5min":10.477,"15min":13.295999999999999},"min":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"max":{"1min":31.138000000000002,"5min":37.701999999999998,"15min":94.343000000000004},"last":9.2010000000000005},{"interface":"front","average":{"1min":10.48,"5min":10.316000000000001,"15min":13.941000000000001},"min":{"1min":4.4029999999999996,"5min":4.4029999999999996,"15min":4.4029999999999996},"max":{"1min":22.327000000000002,"5min":49.045999999999999,"15min":122.054},"last":6.8890000000000002}]},{"osd":1,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":11.297000000000001,"5min":10.913,"15min":10.848000000000001},"min":{"1min":5.1299999999999999,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":22.393000000000001,"5min":24.128,"15min":24.128},"last":8.8870000000000005},{"interface":"front","average":{"1min":13.782999999999999,"5min":13.946999999999999,"15min":27.225000000000001},"min":{"1min":8.0079999999999991,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":37.746000000000002,"5min":37.746000000000002,"15min":387.15600000000001},"last":18.780000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":14.441000000000001,"5min":13.300000000000001,"15min":17.736999999999998},"min":{"1min":8.6199999999999992,"5min":6.5800000000000001,"15min":6.4359999999999999},"max":{"1min":25.774999999999999,"5min":72.988,"15min":156.90299999999999},"last":15.161},{"interface":"front","average":{"1min":17.652999999999999,"5min":16.655000000000001,"15min":19.274000000000001},"min":{"1min":12.808999999999999,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":25.870000000000001,"5min":75.004000000000005,"15min":143.578},"last":16.93}]},{"osd":3,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.574999999999999,"5min":16.600999999999999,"15min":20.175000000000001},"min":{"1min":8.875,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":37.109999999999999,"5min":50.095999999999997,"15min":141.18799999999999},"last":9.9350000000000005},{"interface":"front","average":{"1min":19.908999999999999,"5min":19.231999999999999,"15min":23.358000000000001},"min":{"1min":13.304,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":44.314,"5min":50.887999999999998,"15min":161.32900000000001},"last":14.372999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.367000000000001,"5min":12.712,"15min":14.266999999999999},"min":{"1min":9.8480000000000008,"5min":6.2240000000000002,"15min":6.2240000000000002},"max":{"1min":30.379999999999999,"5min":30.379999999999999,"15min":74.079999999999998},"last":15.728999999999999},{"interface":"front","average":{"1min":13.227,"5min":14.395,"15min":15.266},"min":{"1min":6.782,"5min":6.782,"15min":6.782},"max":{"1min":29.75,"5min":29.75,"15min":58.613999999999997},"last":10.246}]},{"osd":5,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.751000000000001,"5min":14.045999999999999,"15min":16.658999999999999},"min":{"1min":8.8949999999999996,"5min":7.383,"15min":7.383},"max":{"1min":48.072000000000003,"5min":48.072000000000003,"15min":71.757000000000005},"last":12.531000000000001},{"interface":"front","average":{"1min":22.440000000000001,"5min":19.170000000000002,"15min":20.898},"min":{"1min":11.635999999999999,"5min":9.532,"15min":9.532},"max":{"1min":49.463999999999999,"5min":49.463999999999999,"15min":80.119},"last":10.984}]},{"osd":7,"last update":"Mon Oct 30 06:30:21 2023","interfaces":[{"interface":"back","average":{"1min":16.024000000000001,"5min":15.693,"15min":18.390999999999998},"min":{"1min":9.1289999999999996,"5min":8.9610000000000003,"15min":8.9610000000000003},"max":{"1min":27.899000000000001,"5min":29.869,"15min":81.247},"last":13.348000000000001},{"interface":"front","average":{"1min":19.131,"5min":17.087,"15min":18.420999999999999},"min":{"1min":11.939,"5min":8.6310000000000002,"15min":8.6310000000000002},"max":{"1min":32.530000000000001,"5min":32.530000000000001,"15min":70.989000000000004},"last":17.593}]}]},{"osd":3,"up_from":16,"seq":68719476824,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":8.5090000000000003,"5min":10.337,"15min":10.819000000000001},"min":{"1min":4.423,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":18.408999999999999,"5min":48.792000000000002,"15min":48.792000000000002},"last":6.2969999999999997},{"interface":"front","average":{"1min":10.489000000000001,"5min":10.949999999999999,"15min":10.488},"min":{"1min":5.6609999999999996,"5min":4.7220000000000004,"15min":4.7220000000000004},"max":{"1min":19.529,"5min":28.616,"15min":28.616},"last":10.904}]},{"osd":1,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":12.657,"5min":14.374000000000001,"15min":22.675999999999998},"min":{"1min":7.4779999999999998,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":17.920000000000002,"5min":81.552999999999997,"15min":165.76400000000001},"last":20.739000000000001},{"interface":"front","average":{"1min":11.739000000000001,"5min":13.801,"15min":23.318999999999999},"min":{"1min":7.0460000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":25.146999999999998,"5min":82.995000000000005,"15min":190.34200000000001},"last":21.443999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":13.305999999999999,"5min":15.49,"15min":17.448},"min":{"1min":7.7279999999999998,"5min":7.7279999999999998,"15min":7.7279999999999998},"max":{"1min":21.327000000000002,"5min":42.881999999999998,"15min":42.881999999999998},"last":14.335000000000001},{"interface":"front","average":{"1min":14.321,"5min":15.756,"15min":16.718},"min":{"1min":9.7870000000000008,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":20.465,"5min":71.721000000000004,"15min":71.721000000000004},"last":15.315}]},{"osd":4,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":16.334,"5min":17.201000000000001,"15min":19.48},"min":{"1min":8.6609999999999996,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":33.996000000000002,"5min":44.091000000000001,"15min":44.091000000000001},"last":11.678000000000001},{"interface":"front","average":{"1min":10.779999999999999,"5min":11.087,"15min":11.438000000000001},"min":{"1min":7.4119999999999999,"5min":5.7969999999999997,"15min":5.7969999999999997},"max":{"1min":36.658000000000001,"5min":45.231000000000002,"15min":45.231000000000002},"last":10.500999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:30:28 2023","interfaces":[{"interface":"back","average":{"1min":13.082000000000001,"5min":15.159000000000001,"15min":18.158999999999999},"min":{"1min":6.8769999999999998,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":26.584,"5min":57.375999999999998,"15min":57.375999999999998},"last":13.125},{"interface":"front","average":{"1min":16.992000000000001,"5min":15.164,"15min":18.920000000000002},"min":{"1min":8.5969999999999995,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":27.321000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":19.704999999999998}]},{"osd":6,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":13.994,"5min":16.338999999999999,"15min":24.870999999999999},"min":{"1min":9.9049999999999994,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":22.645,"5min":41.771000000000001,"15min":169.05199999999999},"last":13.759},{"interface":"front","average":{"1min":16.689,"5min":15.561,"15min":15.516},"min":{"1min":9.6259999999999994,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":25.077000000000002,"5min":40.128999999999998,"15min":40.128999999999998},"last":17.623999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:30:23 2023","interfaces":[{"interface":"back","average":{"1min":18.183,"5min":19.457999999999998,"15min":18.475999999999999},"min":{"1min":13.298999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":24.167999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":18.373999999999999},{"interface":"front","average":{"1min":18.882000000000001,"5min":20.048999999999999,"15min":26.425999999999998},"min":{"1min":11.368,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":28.706,"5min":55.331000000000003,"15min":170.232},"last":16.178000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476822,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":10.113,"5min":11.311,"15min":22.309000000000001},"min":{"1min":5.6219999999999999,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":24.774999999999999,"5min":91.290999999999997,"15min":258.66199999999998},"last":19.013999999999999},{"interface":"front","average":{"1min":10.486000000000001,"5min":11.595000000000001,"15min":19.727},"min":{"1min":5.6580000000000004,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":23.321999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":22.856999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":11.568,"5min":12.488,"15min":15.036},"min":{"1min":5.3419999999999996,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":23.634,"5min":31.689,"15min":45.936999999999998},"last":9.3119999999999994},{"interface":"front","average":{"1min":14.749000000000001,"5min":12.846,"15min":14.173999999999999},"min":{"1min":8.1359999999999992,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":24.257000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":21.122}]},{"osd":3,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":10.035,"5min":10.699,"15min":16.449999999999999},"min":{"1min":8.1059999999999999,"5min":6.1950000000000003,"15min":6.1950000000000003},"max":{"1min":13.409000000000001,"5min":24.704000000000001,"15min":183.38900000000001},"last":15.654999999999999},{"interface":"front","average":{"1min":14.807,"5min":14.911,"15min":20.347999999999999},"min":{"1min":9.9809999999999999,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":24.312000000000001,"5min":24.317,"15min":189.374},"last":12.167}]},{"osd":4,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":14.207000000000001,"5min":13.550000000000001,"15min":19.155999999999999},"min":{"1min":10.801,"5min":6.9210000000000003,"15min":6.9210000000000003},"max":{"1min":21.907,"5min":31.632000000000001,"15min":186.01400000000001},"last":17.227},{"interface":"front","average":{"1min":15.428000000000001,"5min":14.722,"15min":18.317},"min":{"1min":7.391,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":24.452999999999999,"5min":36.784999999999997,"15min":167.67599999999999},"last":10.997}]},{"osd":5,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":20.239000000000001,"5min":17.667999999999999,"15min":24.262},"min":{"1min":9.2379999999999995,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":65.251999999999995,"5min":65.251999999999995,"15min":215.80500000000001},"last":16.494},{"interface":"front","average":{"1min":18.771999999999998,"5min":16.721,"15min":23.794},"min":{"1min":12.268000000000001,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":34.042000000000002,"5min":34.042000000000002,"15min":187.863},"last":20.213999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":24.710999999999999,"5min":17.227,"15min":23.917000000000002},"min":{"1min":10.429,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":66.673000000000002,"5min":66.673000000000002,"15min":228.81200000000001},"last":14.550000000000001},{"interface":"front","average":{"1min":19.091999999999999,"5min":18.872,"15min":26.523},"min":{"1min":9.5820000000000007,"5min":9.5820000000000007,"15min":9.3710000000000004},"max":{"1min":45.685000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":7.766}]},{"osd":7,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":20.428999999999998,"5min":18.739999999999998,"15min":24.398},"min":{"1min":11.984,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":29.34,"5min":35.898000000000003,"15min":170.43600000000001},"last":21.943999999999999},{"interface":"front","average":{"1min":18.663,"5min":17.48,"15min":21.518000000000001},"min":{"1min":10.481,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":32.140000000000001,"5min":32.140000000000001,"15min":157.00200000000001},"last":13.436999999999999}]}]},{"osd":0,"up_from":0,"seq":68719476823,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":7,"up_from":15,"seq":64424509527,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":14.269,"5min":10.869,"15min":14.519},"min":{"1min":4.4470000000000001,"5min":4.431,"15min":4.4249999999999998},"max":{"1min":109.586,"5min":109.586,"15min":131.477},"last":26.701000000000001},{"interface":"front","average":{"1min":15.268000000000001,"5min":11.801,"15min":14.923},"min":{"1min":6.5730000000000004,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":89.219999999999999,"5min":89.219999999999999,"15min":129.386},"last":27.597000000000001}]},{"osd":1,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":17.631,"5min":12.243,"15min":11.41},"min":{"1min":7.1200000000000001,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":68.385000000000005,"5min":68.385000000000005,"15min":68.385000000000005},"last":15.337},{"interface":"front","average":{"1min":23.324000000000002,"5min":17.375,"15min":18.783999999999999},"min":{"1min":11.907999999999999,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":77.412999999999997,"5min":77.412999999999997,"15min":91.295000000000002},"last":20.016999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":12.904,"5min":12.074,"15min":16.170000000000002},"min":{"1min":5.1470000000000002,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":30.172999999999998,"5min":46.811,"15min":134.21199999999999},"last":28.661999999999999},{"interface":"front","average":{"1min":12.067,"5min":13.321999999999999,"15min":14.196999999999999},"min":{"1min":6.8799999999999999,"5min":5.992,"15min":5.992},"max":{"1min":17.074999999999999,"5min":50.450000000000003,"15min":50.450000000000003},"last":23.748000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":13.842000000000001,"5min":14.667999999999999,"15min":18.667999999999999},"min":{"1min":7.5540000000000003,"5min":6.758,"15min":6.758},"max":{"1min":40.790999999999997,"5min":67.840000000000003,"15min":132.56100000000001},"last":24.710999999999999},{"interface":"front","average":{"1min":16.870999999999999,"5min":16.372,"15min":19.582000000000001},"min":{"1min":10.417,"5min":9.1150000000000002,"15min":6.5780000000000003},"max":{"1min":38.597000000000001,"5min":66.106999999999999,"15min":124.684},"last":52.353000000000002}]},{"osd":4,"last update":"Mon Oct 30 06:30:26 2023","interfaces":[{"interface":"back","average":{"1min":14.448,"5min":14.148,"15min":13.974},"min":{"1min":6.2409999999999997,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":22.405999999999999,"5min":36.552,"15min":36.552},"last":21.093},{"interface":"front","average":{"1min":16.242000000000001,"5min":16.524999999999999,"15min":15.909000000000001},"min":{"1min":10.403,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":25.494,"5min":36.908999999999999,"15min":36.908999999999999},"last":16.446999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:30:26 2023","interfaces":[{"interface":"back","average":{"1min":11.68,"5min":12.712999999999999,"15min":18.053999999999998},"min":{"1min":6.6790000000000003,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":27.515000000000001,"5min":27.515000000000001,"15min":125.792},"last":7.5990000000000002},{"interface":"front","average":{"1min":16.539000000000001,"5min":16.797999999999998,"15min":16.937000000000001},"min":{"1min":9.5640000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":22.021999999999998,"5min":29.338000000000001,"15min":29.338000000000001},"last":22.221}]},{"osd":6,"last update":"Mon Oct 30 06:30:32 2023","interfaces":[{"interface":"back","average":{"1min":20.535,"5min":17.939,"15min":27.425000000000001},"min":{"1min":9.4559999999999995,"5min":7.6020000000000003,"15min":7.6020000000000003},"max":{"1min":59.167000000000002,"5min":59.167000000000002,"15min":327.29599999999999},"last":17.085999999999999},{"interface":"front","average":{"1min":18.082000000000001,"5min":15.004,"15min":27.614000000000001},"min":{"1min":8.1289999999999996,"5min":7.5499999999999998,"15min":7.5499999999999998},"max":{"1min":57.106999999999999,"5min":57.106999999999999,"15min":323.82400000000001},"last":22.526}]}]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":770048,"data_stored":726293,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":11.613,"5min":10.550000000000001,"15min":13.574},"min":{"1min":5.2430000000000003,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":51.149000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":7.343},{"interface":"front","average":{"1min":14.231,"5min":11.051,"15min":15.044},"min":{"1min":6.407,"5min":5.6559999999999997,"15min":5.6559999999999997},"max":{"1min":50.776000000000003,"5min":50.776000000000003,"15min":88.816000000000003},"last":9.843}]},{"osd":1,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":15.244,"5min":14.02,"15min":16.43},"min":{"1min":7.3220000000000001,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":63.587000000000003,"5min":63.587000000000003,"15min":87.515000000000001},"last":17.25},{"interface":"front","average":{"1min":17.096,"5min":14.686,"15min":16.904},"min":{"1min":6.8099999999999996,"5min":6.5549999999999997,"15min":6.21},"max":{"1min":65.349000000000004,"5min":65.349000000000004,"15min":85.832999999999998},"last":10.548999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":15.500999999999999,"5min":15.102,"15min":17.276},"min":{"1min":8.9269999999999996,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":36.207000000000001,"5min":36.207000000000001,"15min":113.943},"last":12.483000000000001},{"interface":"front","average":{"1min":10.619999999999999,"5min":11.005000000000001,"15min":13.981},"min":{"1min":5.7300000000000004,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":26.721,"5min":33.826000000000001,"15min":80.004000000000005},"last":8.1349999999999998}]},{"osd":3,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":14.715,"5min":15.676,"15min":19.786000000000001},"min":{"1min":6.4770000000000003,"5min":6.2889999999999997,"15min":6.2889999999999997},"max":{"1min":23.344000000000001,"5min":87.866,"15min":87.866},"last":11.255000000000001},{"interface":"front","average":{"1min":15.343999999999999,"5min":16.809000000000001,"15min":21.571000000000002},"min":{"1min":8.6069999999999993,"5min":7.8390000000000004,"15min":7.8390000000000004},"max":{"1min":27.131,"5min":89.334000000000003,"15min":91.183000000000007},"last":13.109999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":19.757000000000001,"5min":14.317,"15min":13.263999999999999},"min":{"1min":11.198,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":41.762999999999998,"5min":41.762999999999998,"15min":41.762999999999998},"last":14.337999999999999},{"interface":"front","average":{"1min":14.374000000000001,"5min":15.544,"15min":16.638000000000002},"min":{"1min":6.1589999999999998,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":42.552,"5min":42.552,"15min":59.609000000000002},"last":8.8800000000000008}]},{"osd":6,"last update":"Mon Oct 30 06:30:25 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":15.319000000000001,"15min":15.853999999999999},"min":{"1min":8.9049999999999994,"5min":7.6980000000000004,"15min":7.6980000000000004},"max":{"1min":45.054000000000002,"5min":46.561999999999998,"15min":66.765000000000001},"last":16.457000000000001},{"interface":"front","average":{"1min":17.545000000000002,"5min":18.350999999999999,"15min":20.289000000000001},"min":{"1min":6.6079999999999997,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":45.877000000000002,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.723000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:30:18 2023","interfaces":[{"interface":"back","average":{"1min":18.405000000000001,"5min":17.689,"15min":18.661000000000001},"min":{"1min":10.512,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":30.574999999999999,"5min":35.192,"15min":64.677000000000007},"last":11.840999999999999},{"interface":"front","average":{"1min":16.315999999999999,"5min":16.181000000000001,"15min":17.646999999999998},"min":{"1min":7.9960000000000004,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":25.489000000000001,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.949999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":328453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":9.6120000000000001,"5min":9.7490000000000006,"15min":9.4559999999999995},"min":{"1min":4.6109999999999998,"5min":4.6109999999999998,"15min":4.5410000000000004},"max":{"1min":16.666,"5min":26.984999999999999,"15min":26.984999999999999},"last":18.545999999999999},{"interface":"front","average":{"1min":9.4499999999999993,"5min":10.565,"15min":10.56},"min":{"1min":6.0919999999999996,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":22.981000000000002,"5min":25.934999999999999,"15min":25.934999999999999},"last":9.0739999999999998}]},{"osd":1,"last update":"Mon Oct 30 06:30:24 2023","interfaces":[{"interface":"back","average":{"1min":11.544,"5min":12.411,"15min":11.784000000000001},"min":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"max":{"1min":22.372,"5min":25.122,"15min":25.122},"last":15.119999999999999},{"interface":"front","average":{"1min":14.856999999999999,"5min":13.547000000000001,"15min":13.951000000000001},"min":{"1min":8.2110000000000003,"5min":7.218,"15min":6.609},"max":{"1min":20.957999999999998,"5min":30.751999999999999,"15min":30.751999999999999},"last":7.7439999999999998}]},{"osd":2,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":16.553999999999998,"5min":14.986000000000001,"15min":12.974},"min":{"1min":8.7919999999999998,"5min":7.4530000000000003,"15min":6.6200000000000001},"max":{"1min":23.042000000000002,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.433},{"interface":"front","average":{"1min":13.771000000000001,"5min":14.698,"15min":15.416},"min":{"1min":8.1609999999999996,"5min":7.907,"15min":7.907},"max":{"1min":27.091000000000001,"5min":34.122999999999998,"15min":34.122999999999998},"last":14.548}]},{"osd":3,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":13.349,"5min":10.861000000000001,"15min":11.130000000000001},"min":{"1min":6.1079999999999997,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":32.804000000000002,"5min":66.722999999999999,"15min":66.722999999999999},"last":6.9880000000000004},{"interface":"front","average":{"1min":16.425999999999998,"5min":16.138999999999999,"15min":16.09},"min":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":24.254000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":15.712}]},{"osd":5,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":18.224,"5min":17.117000000000001,"15min":18.774000000000001},"min":{"1min":10.023,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":50.207000000000001,"5min":50.207000000000001,"15min":64.474999999999994},"last":10.503},{"interface":"front","average":{"1min":15.733000000000001,"5min":14.039,"15min":15.487},"min":{"1min":7.0709999999999997,"5min":6.6440000000000001,"15min":5.7770000000000001},"max":{"1min":48.356000000000002,"5min":48.356000000000002,"15min":75.980000000000004},"last":9.7859999999999996}]},{"osd":6,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":21.312999999999999,"5min":18.829000000000001,"15min":18.795000000000002},"min":{"1min":10.291,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":60.633000000000003,"5min":60.633000000000003,"15min":60.633000000000003},"last":11.81},{"interface":"front","average":{"1min":17.771999999999998,"5min":14.288,"15min":14.113},"min":{"1min":7.343,"5min":7.343,"15min":7.343},"max":{"1min":48.267000000000003,"5min":48.267000000000003,"15min":48.267000000000003},"last":13.185}]},{"osd":7,"last update":"Mon Oct 30 06:30:19 2023","interfaces":[{"interface":"back","average":{"1min":17.266999999999999,"5min":16.823,"15min":17.846},"min":{"1min":12.065,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":27.675999999999998,"5min":41.783999999999999,"15min":74.977999999999994},"last":13.901},{"interface":"front","average":{"1min":21.209,"5min":19.164999999999999,"15min":21.291},"min":{"1min":12.215999999999999,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":39.973999999999997,"5min":44.450000000000003,"15min":83.138000000000005},"last":19.443000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:11.256 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [5, 0, 7, 4] 2023-10-30T06:31:11.256 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '5', '0', '7', '4'] 2023-10-30T06:31:11.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.2 5 0 7 4 2023-10-30T06:31:12.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-10-30T06:31:13.188 INFO:teuthology.orchestra.run.smithi079.stderr:set 2.2 pg_upmap_items mapping to [5->0,7->4] 2023-10-30T06:31:14.266 INFO:teuthology.orchestra.run.smithi079.stderr:noscrub is unset 2023-10-30T06:31:14.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-10-30T06:31:14.708 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:46.589 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.711 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:46.592 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.731 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:02:46.612 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.758 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:41.093 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.759 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:41.095 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.760 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:28.533 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.762 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:28.535 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.774 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:41.109 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:14.783 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:28.555 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:15.863 INFO:teuthology.orchestra.run.smithi079.stderr:nodeep-scrub is unset 2023-10-30T06:31:18.201 INFO:tasks.thrashosds.thrasher:inning osd 2023-10-30T06:31:18.201 INFO:tasks.thrashosds.thrasher:Adding osd 0 2023-10-30T06:31:18.202 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2023-10-30T06:31:18.940 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:31:18.936+0000 ff6f640 -1 mon.a@0(leader).osd e129 definitely_dead 0 2023-10-30T06:31:19.962 INFO:teuthology.orchestra.run.smithi079.stderr:marked in osd.0. 2023-10-30T06:31:19.977 INFO:tasks.thrashosds.thrasher:Added osd 0 2023-10-30T06:31:19.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2023-10-30T06:31:20.267 INFO:teuthology.orchestra.run.smithi079.stdout:mon.a: {} 2023-10-30T06:31:20.267 INFO:teuthology.orchestra.run.smithi079.stderr:mon.a: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2023-10-30T06:31:20.378 INFO:teuthology.orchestra.run.smithi079.stdout:mon.b: {} 2023-10-30T06:31:20.379 INFO:teuthology.orchestra.run.smithi079.stderr:mon.b: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2023-10-30T06:31:20.489 INFO:teuthology.orchestra.run.smithi079.stdout:mon.c: {} 2023-10-30T06:31:20.490 INFO:teuthology.orchestra.run.smithi079.stderr:mon.c: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2023-10-30T06:31:20.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-10-30T06:31:22.270 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.0 to 1 (10000) 2023-10-30T06:31:22.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2023-10-30T06:31:23.355 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.0 primary-affinity to 1 (8655362) 2023-10-30T06:31:23.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-10-30T06:31:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.1 to 1 (10000) 2023-10-30T06:31:24.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2023-10-30T06:31:26.006 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.1 primary-affinity to 1 (8655362) 2023-10-30T06:31:26.019 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-10-30T06:31:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.2 to 1 (10000) 2023-10-30T06:31:27.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2023-10-30T06:31:28.110 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.2 primary-affinity to 1 (8655362) 2023-10-30T06:31:28.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-10-30T06:31:29.247 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.3 to 1 (10000) 2023-10-30T06:31:29.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2023-10-30T06:31:29.764 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:01.644 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.769 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:01.650 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.793 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:01.673 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.808 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.143 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.813 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.148 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.833 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.168 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.837 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.609 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.842 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.615 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:29.862 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.634 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.147 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.482 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.149 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.484 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.157 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:01:56.493 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.163 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:02.044 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.164 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:02.045 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.173 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:02.054 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.215 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.987 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.216 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.988 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.226 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:00:43.998 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:30.954 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.3 primary-affinity to 1 (8655362) 2023-10-30T06:31:30.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-10-30T06:31:32.013 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.4 to 1 (10000) 2023-10-30T06:31:32.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2023-10-30T06:31:33.126 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.4 primary-affinity to 1 (8655362) 2023-10-30T06:31:33.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2023-10-30T06:31:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.5 to 1 (10000) 2023-10-30T06:31:34.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2023-10-30T06:31:35.797 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.5 primary-affinity to 1 (8655362) 2023-10-30T06:31:35.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-10-30T06:31:37.373 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.6 to 1 (10000) 2023-10-30T06:31:37.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2023-10-30T06:31:38.381 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.6 primary-affinity to 1 (8655362) 2023-10-30T06:31:38.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-10-30T06:31:39.588 INFO:teuthology.orchestra.run.smithi079.stderr:reweighted osd.7 to 1 (10000) 2023-10-30T06:31:39.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2023-10-30T06:31:41.039 INFO:teuthology.orchestra.run.smithi079.stderr:set osd.7 primary-affinity to 1 (8655362) 2023-10-30T06:31:41.053 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2023-10-30T06:31:41.054 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2023-10-30T06:31:41.054 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2023-10-30T06:31:41.054 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2023-10-30T06:31:41.054 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-30T06:31:41.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:31:41.640 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:41.641 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":147,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:31:40.969641+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:31:19.833616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9800000190734863,"score_stable":3.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":3.5,"raw_score_stable":3.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,0]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:31:41.654 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-30T06:31:41.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:31:42.239 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:42.240 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":147,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:31:40.969641+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:31:19.833616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9800000190734863,"score_stable":3.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":3.5,"raw_score_stable":3.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,0]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:31:42.252 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-30T06:31:42.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-30T06:31:42.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-30T06:31:42.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-30T06:31:42.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-30T06:31:42.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-30T06:31:42.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-30T06:31:42.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-30T06:31:42.600 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:42.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:31:42.623 INFO:teuthology.orchestra.run.smithi079.stdout:77309411422 2023-10-30T06:31:42.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:31:42.625 INFO:teuthology.orchestra.run.smithi079.stdout:68719476832 2023-10-30T06:31:42.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T06:31:42.640 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:42.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:31:42.664 INFO:teuthology.orchestra.run.smithi079.stdout:68719476832 2023-10-30T06:31:42.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:31:42.686 INFO:teuthology.orchestra.run.smithi079.stdout:68719476830 2023-10-30T06:31:42.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T06:31:42.703 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:42.704 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T06:31:42.719 INFO:teuthology.orchestra.run.smithi079.stdout:68719476831 2023-10-30T06:31:42.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T06:31:43.354 INFO:teuthology.orchestra.run.smithi079.stdout:64424509534 2023-10-30T06:31:43.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.4 2023-10-30T06:31:43.545 INFO:teuthology.orchestra.run.smithi079.stdout:68719476830 2023-10-30T06:31:43.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476832 got 68719476830 for osd.3 2023-10-30T06:31:43.679 INFO:teuthology.orchestra.run.smithi079.stdout:64424509534 2023-10-30T06:31:43.682 INFO:teuthology.orchestra.run.smithi079.stdout:68719476830 2023-10-30T06:31:43.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.5 2023-10-30T06:31:43.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476832 got 68719476830 for osd.6 2023-10-30T06:31:43.853 INFO:teuthology.orchestra.run.smithi079.stdout:77309411421 2023-10-30T06:31:43.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411422 got 77309411421 for osd.1 2023-10-30T06:31:43.871 INFO:teuthology.orchestra.run.smithi079.stdout:68719476828 2023-10-30T06:31:43.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476830 got 68719476828 for osd.2 2023-10-30T06:31:43.971 INFO:teuthology.orchestra.run.smithi079.stdout:64424509534 2023-10-30T06:31:43.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.7 2023-10-30T06:31:44.087 INFO:teuthology.orchestra.run.smithi079.stdout:68719476829 2023-10-30T06:31:44.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476831 got 68719476829 for osd.0 2023-10-30T06:31:44.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:31:44.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:31:44.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:31:44.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-30T06:31:44.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:31:44.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-30T06:31:44.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-30T06:31:45.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-30T06:31:45.250 INFO:teuthology.orchestra.run.smithi079.stdout:64424509534 2023-10-30T06:31:45.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.4 2023-10-30T06:31:45.481 INFO:teuthology.orchestra.run.smithi079.stdout:77309411421 2023-10-30T06:31:45.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411422 got 77309411421 for osd.1 2023-10-30T06:31:45.525 INFO:teuthology.orchestra.run.smithi079.stdout:68719476830 2023-10-30T06:31:45.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476832 got 68719476830 for osd.3 2023-10-30T06:31:45.730 INFO:teuthology.orchestra.run.smithi079.stdout:64424509534 2023-10-30T06:31:45.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.5 2023-10-30T06:31:45.845 INFO:teuthology.orchestra.run.smithi079.stdout:68719476830 2023-10-30T06:31:45.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476830 got 68719476830 for osd.2 2023-10-30T06:31:45.861 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:45.894 INFO:teuthology.orchestra.run.smithi079.stdout:68719476832 2023-10-30T06:31:45.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476832 got 68719476832 for osd.6 2023-10-30T06:31:45.909 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:45.991 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:46.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509535 for osd.7 2023-10-30T06:31:46.009 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:46.086 INFO:teuthology.orchestra.run.smithi079.stdout:68719476831 2023-10-30T06:31:46.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476831 got 68719476831 for osd.0 2023-10-30T06:31:46.099 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:46.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-30T06:31:46.495 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-30T06:31:46.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-30T06:31:46.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-30T06:31:46.993 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:47.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509535 for osd.4 2023-10-30T06:31:47.007 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:47.079 INFO:teuthology.orchestra.run.smithi079.stdout:77309411423 2023-10-30T06:31:47.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411422 got 77309411423 for osd.1 2023-10-30T06:31:47.092 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:47.165 INFO:teuthology.orchestra.run.smithi079.stdout:68719476832 2023-10-30T06:31:47.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476832 got 68719476832 for osd.3 2023-10-30T06:31:47.178 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:47.398 INFO:teuthology.orchestra.run.smithi079.stdout:64424509535 2023-10-30T06:31:47.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509535 for osd.5 2023-10-30T06:31:47.411 DEBUG:teuthology.parallel:result is None 2023-10-30T06:31:47.412 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-10-30T06:31:47.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:47.969 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:47.970 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:47.980 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2023-10-30T06:31:47.333272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222088,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752632,"statfs":{"total":773094113280,"available":772866695168,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.565869+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:39.565869+0000","last_peered":"2023-10-30T06:31:39.565869+0000","last_clean":"2023-10-30T06:31:39.565869+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:39.565869+0000","last_undegraded":"2023-10-30T06:31:39.565869+0000","last_fullsized":"2023-10-30T06:31:39.565869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:07.024520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.561116+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:39.561116+0000","last_peered":"2023-10-30T06:31:39.561116+0000","last_clean":"2023-10-30T06:31:39.561116+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:39.561116+0000","last_undegraded":"2023-10-30T06:31:39.561116+0000","last_fullsized":"2023-10-30T06:31:39.561116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.558612+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:39.558612+0000","last_peered":"2023-10-30T06:31:39.558612+0000","last_clean":"2023-10-30T06:31:39.558612+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:39.558612+0000","last_undegraded":"2023-10-30T06:31:39.558612+0000","last_fullsized":"2023-10-30T06:31:39.558612+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.568366+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:39.568366+0000","last_peered":"2023-10-30T06:31:39.568366+0000","last_clean":"2023-10-30T06:31:39.568366+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:39.568366+0000","last_undegraded":"2023-10-30T06:31:39.568366+0000","last_fullsized":"2023-10-30T06:31:39.568366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":238,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059449+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:40.059449+0000","last_peered":"2023-10-30T06:31:40.059449+0000","last_clean":"2023-10-30T06:31:40.059449+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:40.059449+0000","last_undegraded":"2023-10-30T06:31:40.059449+0000","last_fullsized":"2023-10-30T06:31:40.059449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411423,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":14.305999999999999},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":6.8099999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":10.478999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":15.231999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":19.640999999999998},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":8.5340000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":24.23},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":22.555}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":12.601000000000001},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.504}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":18.504999999999999},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":21.832999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":20.585999999999999},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":15.942}]}]},{"osd":6,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.0289999999999999},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":7.7809999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":5.7800000000000002},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":8.516}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":17.117999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":14.693}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.539999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.115}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":18.265000000000001},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":13.993}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.795},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":11.757999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":16.428000000000001},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":19.209}]}]},{"osd":3,"up_from":16,"seq":68719476833,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":17.512},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":9.8740000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":11.34},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":14.759}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":19.401},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.451000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":14.125},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.5809999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":12.734},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":24.379999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":16.585999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":20.986000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":22.934000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":21.306999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":5.4809999999999999},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.2430000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":12.031000000000001},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.8399999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":12.385999999999999},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":9.1430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":15.108000000000001},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":13.336}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":18.565999999999999},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.473000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":13.071999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":15.848000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":13.942},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":17.664999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476832,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.8819999999999997},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.654}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":15.58},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":18.327999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":25.632999999999999},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":27.428000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":11.710000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":17.379000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":21.407},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":22.925000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":24.088000000000001},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":14.922000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":28.731000000000002},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":19.280999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":16.181999999999999},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":9.4299999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":10.055999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":11.898999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":11.282},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":7.1280000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":15.02},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":10.645}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":15.632999999999999},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":17.603999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":8.7289999999999992},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":14.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":18.515999999999998},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":12.541}]}]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":11.18},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.141999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":9.3789999999999996},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":6.7089999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":8.2970000000000006},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":7.3780000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":12.380000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.725999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":11.788},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.030000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":14.993},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.068}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":14.287000000000001},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.873999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.0309999999999997},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.0210000000000008}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":9.6539999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.15}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.103000000000002},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.635999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":9.9339999999999993},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.567}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":14.938000000000001},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.257999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.367000000000001},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.757000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.808},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":15.552}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:47.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:48.569 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:48.570 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:48.581 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2023-10-30T06:31:47.333272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222088,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752632,"statfs":{"total":773094113280,"available":772866695168,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.565869+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:39.565869+0000","last_peered":"2023-10-30T06:31:39.565869+0000","last_clean":"2023-10-30T06:31:39.565869+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:39.565869+0000","last_undegraded":"2023-10-30T06:31:39.565869+0000","last_fullsized":"2023-10-30T06:31:39.565869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:07.024520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.561116+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:39.561116+0000","last_peered":"2023-10-30T06:31:39.561116+0000","last_clean":"2023-10-30T06:31:39.561116+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:39.561116+0000","last_undegraded":"2023-10-30T06:31:39.561116+0000","last_fullsized":"2023-10-30T06:31:39.561116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.558612+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:39.558612+0000","last_peered":"2023-10-30T06:31:39.558612+0000","last_clean":"2023-10-30T06:31:39.558612+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:39.558612+0000","last_undegraded":"2023-10-30T06:31:39.558612+0000","last_fullsized":"2023-10-30T06:31:39.558612+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.568366+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:39.568366+0000","last_peered":"2023-10-30T06:31:39.568366+0000","last_clean":"2023-10-30T06:31:39.568366+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:39.568366+0000","last_undegraded":"2023-10-30T06:31:39.568366+0000","last_fullsized":"2023-10-30T06:31:39.568366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":238,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059449+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:40.059449+0000","last_peered":"2023-10-30T06:31:40.059449+0000","last_clean":"2023-10-30T06:31:40.059449+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:40.059449+0000","last_undegraded":"2023-10-30T06:31:40.059449+0000","last_fullsized":"2023-10-30T06:31:40.059449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411423,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":14.305999999999999},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":6.8099999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":10.478999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":15.231999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":19.640999999999998},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":8.5340000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":24.23},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":22.555}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":12.601000000000001},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.504}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":18.504999999999999},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":21.832999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":20.585999999999999},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":15.942}]}]},{"osd":6,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.0289999999999999},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":7.7809999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":5.7800000000000002},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":8.516}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":17.117999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":14.693}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.539999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.115}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":18.265000000000001},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":13.993}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.795},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":11.757999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":16.428000000000001},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":19.209}]}]},{"osd":3,"up_from":16,"seq":68719476833,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":17.512},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":9.8740000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":11.34},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":14.759}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":19.401},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.451000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":14.125},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.5809999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":12.734},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":24.379999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":16.585999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":20.986000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":22.934000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":21.306999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":5.4809999999999999},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.2430000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":12.031000000000001},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.8399999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":12.385999999999999},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":9.1430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":15.108000000000001},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":13.336}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":18.565999999999999},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.473000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":13.071999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":15.848000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":13.942},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":17.664999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476832,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.8819999999999997},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.654}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":15.58},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":18.327999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":25.632999999999999},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":27.428000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":11.710000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":17.379000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":21.407},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":22.925000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":24.088000000000001},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":14.922000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":28.731000000000002},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":19.280999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":16.181999999999999},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":9.4299999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":10.055999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":11.898999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":11.282},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":7.1280000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":15.02},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":10.645}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":15.632999999999999},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":17.603999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":8.7289999999999992},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":14.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":18.515999999999998},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":12.541}]}]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":11.18},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.141999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":9.3789999999999996},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":6.7089999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":8.2970000000000006},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":7.3780000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":12.380000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.725999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":11.788},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.030000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":14.993},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.068}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":14.287000000000001},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.873999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.0309999999999997},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.0210000000000008}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":9.6539999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.15}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.103000000000002},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.635999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":9.9339999999999993},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.567}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":14.938000000000001},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.257999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.367000000000001},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.757000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.808},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":15.552}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:48.583 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-10-30T06:31:48.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2023-10-30T06:31:49.333 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-30T06:31:49.343 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-30T06:31:49.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:49.920 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:49.921 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:49.932 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2023-10-30T06:31:49.333557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222088,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752632,"statfs":{"total":773094113280,"available":772866695168,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.795375"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.565869+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:39.565869+0000","last_peered":"2023-10-30T06:31:39.565869+0000","last_clean":"2023-10-30T06:31:39.565869+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:39.565869+0000","last_undegraded":"2023-10-30T06:31:39.565869+0000","last_fullsized":"2023-10-30T06:31:39.565869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:07.024520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.561116+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:39.561116+0000","last_peered":"2023-10-30T06:31:39.561116+0000","last_clean":"2023-10-30T06:31:39.561116+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:39.561116+0000","last_undegraded":"2023-10-30T06:31:39.561116+0000","last_fullsized":"2023-10-30T06:31:39.561116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.558612+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:39.558612+0000","last_peered":"2023-10-30T06:31:39.558612+0000","last_clean":"2023-10-30T06:31:39.558612+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:39.558612+0000","last_undegraded":"2023-10-30T06:31:39.558612+0000","last_fullsized":"2023-10-30T06:31:39.558612+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.568366+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:39.568366+0000","last_peered":"2023-10-30T06:31:39.568366+0000","last_clean":"2023-10-30T06:31:39.568366+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:39.568366+0000","last_undegraded":"2023-10-30T06:31:39.568366+0000","last_fullsized":"2023-10-30T06:31:39.568366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":238,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059449+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:40.059449+0000","last_peered":"2023-10-30T06:31:40.059449+0000","last_clean":"2023-10-30T06:31:40.059449+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:40.059449+0000","last_undegraded":"2023-10-30T06:31:40.059449+0000","last_fullsized":"2023-10-30T06:31:40.059449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411424,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":5.3479999999999999},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":6.4930000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":9.6219999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":12.946999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":17.952000000000002},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":8.6880000000000006}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":19.100000000000001},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":10.791}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":14.907999999999999},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.363}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":14.294},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":16.931000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":20.425000000000001},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":16.064}]}]},{"osd":6,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.0289999999999999},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":7.7809999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":5.7800000000000002},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":8.516}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":17.117999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":14.693}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.539999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.115}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":18.265000000000001},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":13.993}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.795},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":11.757999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":16.428000000000001},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":19.209}]}]},{"osd":3,"up_from":16,"seq":68719476833,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":17.512},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":9.8740000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":11.34},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":14.759}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":19.401},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.451000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":14.125},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.5809999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":12.734},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":24.379999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":16.585999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":20.986000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":22.934000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":21.306999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":5.4809999999999999},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.2430000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":12.031000000000001},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.8399999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":12.385999999999999},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":9.1430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":15.108000000000001},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":13.336}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":18.565999999999999},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.473000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":13.071999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":15.848000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":13.942},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":17.664999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476832,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.8819999999999997},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.654}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":15.58},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":18.327999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":25.632999999999999},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":27.428000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":11.710000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":17.379000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":21.407},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":22.925000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":24.088000000000001},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":14.922000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":28.731000000000002},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":19.280999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":16.181999999999999},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":9.4299999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":10.055999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":11.898999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":11.282},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":7.1280000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":15.02},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":10.645}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":15.632999999999999},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":17.603999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":8.7289999999999992},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":14.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":18.515999999999998},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":12.541}]}]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":11.18},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.141999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":9.3789999999999996},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":6.7089999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":8.2970000000000006},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":7.3780000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":12.380000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.725999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":11.788},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.030000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":14.993},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.068}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":14.287000000000001},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.873999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.0309999999999997},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.0210000000000008}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":9.6539999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.15}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.103000000000002},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.635999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":9.9339999999999993},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.567}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":14.938000000000001},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.257999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.367000000000001},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.757000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.808},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":15.552}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:49.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:50.362 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:22.243 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.366 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:22.247 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.368 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:16.703 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.373 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:16.708 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.394 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:16.728 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.408 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:22.288 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.413 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:04.186 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.416 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:04.189 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.442 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:04.214 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:31:50.515 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:50.516 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:50.527 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2023-10-30T06:31:49.333557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222088,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752632,"statfs":{"total":773094113280,"available":772866695168,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.795375"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.565869+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:39.565869+0000","last_peered":"2023-10-30T06:31:39.565869+0000","last_clean":"2023-10-30T06:31:39.565869+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:39.565869+0000","last_undegraded":"2023-10-30T06:31:39.565869+0000","last_fullsized":"2023-10-30T06:31:39.565869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:07.024520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.561116+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:39.561116+0000","last_peered":"2023-10-30T06:31:39.561116+0000","last_clean":"2023-10-30T06:31:39.561116+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:39.561116+0000","last_undegraded":"2023-10-30T06:31:39.561116+0000","last_fullsized":"2023-10-30T06:31:39.561116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.558612+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:39.558612+0000","last_peered":"2023-10-30T06:31:39.558612+0000","last_clean":"2023-10-30T06:31:39.558612+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:39.558612+0000","last_undegraded":"2023-10-30T06:31:39.558612+0000","last_fullsized":"2023-10-30T06:31:39.558612+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.568366+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:39.568366+0000","last_peered":"2023-10-30T06:31:39.568366+0000","last_clean":"2023-10-30T06:31:39.568366+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:39.568366+0000","last_undegraded":"2023-10-30T06:31:39.568366+0000","last_fullsized":"2023-10-30T06:31:39.568366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":238,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059449+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:40.059449+0000","last_peered":"2023-10-30T06:31:40.059449+0000","last_clean":"2023-10-30T06:31:40.059449+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:40.059449+0000","last_undegraded":"2023-10-30T06:31:40.059449+0000","last_fullsized":"2023-10-30T06:31:40.059449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411424,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":5.3479999999999999},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":6.4930000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":9.6219999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":12.946999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":17.952000000000002},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":8.6880000000000006}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":19.100000000000001},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":10.791}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":14.907999999999999},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.363}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":14.294},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":16.931000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":20.425000000000001},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":16.064}]}]},{"osd":6,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.0289999999999999},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":7.7809999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":5.7800000000000002},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":8.516}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":17.117999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":14.693}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.539999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.115}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":18.265000000000001},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":13.993}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.795},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":11.757999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":16.428000000000001},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":19.209}]}]},{"osd":3,"up_from":16,"seq":68719476833,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":17.512},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":9.8740000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":11.34},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":14.759}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":19.401},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.451000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":14.125},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.5809999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":12.734},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":24.379999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":16.585999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":20.986000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":22.934000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":21.306999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":5.4809999999999999},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.2430000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":12.031000000000001},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.8399999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":12.385999999999999},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":9.1430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":15.108000000000001},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":13.336}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":18.565999999999999},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.473000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":13.071999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":15.848000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":13.942},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":17.664999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476832,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.8819999999999997},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.654}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":15.58},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":18.327999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":25.632999999999999},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":27.428000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":11.710000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":17.379000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":21.407},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":22.925000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":24.088000000000001},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":14.922000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":28.731000000000002},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":19.280999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":16.181999999999999},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":9.4299999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":10.055999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":11.898999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":11.282},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":7.1280000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":15.02},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":10.645}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":15.632999999999999},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":17.603999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":8.7289999999999992},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":14.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":18.515999999999998},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":12.541}]}]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":11.18},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.141999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":9.3789999999999996},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":6.7089999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":8.2970000000000006},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":7.3780000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":12.380000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.725999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":11.788},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.030000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":14.993},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.068}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":14.287000000000001},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.873999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.0309999999999997},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.0210000000000008}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":9.6539999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.15}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.103000000000002},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.635999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":9.9339999999999993},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.567}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":14.938000000000001},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.257999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.367000000000001},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.757000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.808},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":15.552}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:50.530 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-30T06:31:50.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:51.153 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:51.154 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:51.165 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2023-10-30T06:31:49.333557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222088,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752632,"statfs":{"total":773094113280,"available":772866695168,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.795375"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":64,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.565869+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:39.565869+0000","last_peered":"2023-10-30T06:31:39.565869+0000","last_clean":"2023-10-30T06:31:39.565869+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:39.565869+0000","last_undegraded":"2023-10-30T06:31:39.565869+0000","last_fullsized":"2023-10-30T06:31:39.565869+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T18:05:07.024520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.561116+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:39.561116+0000","last_peered":"2023-10-30T06:31:39.561116+0000","last_clean":"2023-10-30T06:31:39.561116+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:39.561116+0000","last_undegraded":"2023-10-30T06:31:39.561116+0000","last_fullsized":"2023-10-30T06:31:39.561116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:25:05.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.558612+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:39.558612+0000","last_peered":"2023-10-30T06:31:39.558612+0000","last_clean":"2023-10-30T06:31:39.558612+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:39.558612+0000","last_undegraded":"2023-10-30T06:31:39.558612+0000","last_fullsized":"2023-10-30T06:31:39.558612+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:27:37.626489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":50,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:39.568366+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:39.568366+0000","last_peered":"2023-10-30T06:31:39.568366+0000","last_clean":"2023-10-30T06:31:39.568366+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:39.568366+0000","last_undegraded":"2023-10-30T06:31:39.568366+0000","last_fullsized":"2023-10-30T06:31:39.568366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T06:37:30.425125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"24'76","reported_seq":238,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059449+0000","last_change":"2023-10-30T06:24:14.658360+0000","last_active":"2023-10-30T06:31:40.059449+0000","last_peered":"2023-10-30T06:31:40.059449+0000","last_clean":"2023-10-30T06:31:40.059449+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:40.059449+0000","last_undegraded":"2023-10-30T06:31:40.059449+0000","last_fullsized":"2023-10-30T06:31:40.059449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:10.404444+0000","last_clean_scrub_stamp":"2023-10-30T06:24:10.404444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:31:27.178386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411424,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":5.3479999999999999},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":6.4930000000000003}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":9.6219999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":12.946999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":17.952000000000002},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":8.6880000000000006}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":19.100000000000001},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":10.791}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":14.907999999999999},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.363}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":14.294},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":16.931000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":20.425000000000001},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":16.064}]}]},{"osd":6,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.0289999999999999},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":7.7809999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":5.7800000000000002},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":8.516}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":17.117999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":14.693}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.539999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.115}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":18.265000000000001},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":13.993}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.795},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":11.757999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":16.428000000000001},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":19.209}]}]},{"osd":3,"up_from":16,"seq":68719476833,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":17.512},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":9.8740000000000006}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":11.34},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":14.759}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":19.401},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":13.451000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":14.125},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":8.5809999999999995}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":12.734},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":24.379999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":16.585999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":20.986000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":22.934000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":21.306999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":5.4809999999999999},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.2430000000000003}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":12.031000000000001},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.8399999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":12.385999999999999},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":9.1430000000000007}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":15.108000000000001},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":13.336}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":18.565999999999999},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.473000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":13.071999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":15.848000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":13.942},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":17.664999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476832,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.8819999999999997},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.654}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":15.58},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":18.327999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":25.632999999999999},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":27.428000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":11.710000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":17.379000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":21.407},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":22.925000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":24.088000000000001},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":14.922000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":28.731000000000002},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":19.280999999999999}]}]},{"osd":7,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":16.181999999999999},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":9.4299999999999997}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":10.055999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":11.898999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":11.282},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":7.1280000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":15.02},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":10.645}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":15.632999999999999},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":17.603999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":8.7289999999999992},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":14.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":18.515999999999998},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":12.541}]}]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":11.18},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":10.141999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":9.3789999999999996},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":6.7089999999999996}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":8.2970000000000006},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":7.3780000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":12.380000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.725999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":11.788},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.030000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":14.993},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":13.068}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":14.287000000000001},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":17.873999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.0309999999999997},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.0210000000000008}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":9.6539999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.15}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":16.103000000000002},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.635999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":9.9339999999999993},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.567}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":14.938000000000001},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.257999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.367000000000001},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":16.757000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.808},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":15.552}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:51.167 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-30T06:31:51.884 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:51.884 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":147,"fsid":"09a24e68-832d-4fbc-9604-48e808368d81","created":"2023-10-30T06:23:23.243533+0000","modified":"2023-10-30T06:31:40.969641+0000","last_up_change":"2023-10-30T06:24:12.541257+0000","last_in_change":"2023-10-30T06:31:19.833616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-30T06:24:09.653526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-30T06:24:17.127645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9800000190734863,"score_stable":3.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":3.5,"raw_score_stable":3.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6e79cc30-6aa7-489d-8806-48c107644b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6811","nonce":2677821092}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6813","nonce":2677821092}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6817","nonce":2677821092}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":2677821092},{"type":"v1","addr":"172.21.15.79:6815","nonce":2677821092}]},"public_addr":"172.21.15.79:6811/2677821092","cluster_addr":"172.21.15.79:6813/2677821092","heartbeat_back_addr":"172.21.15.79:6817/2677821092","heartbeat_front_addr":"172.21.15.79:6815/2677821092","state":["exists","up"]},{"osd":1,"uuid":"c8a16891-3cdc-487f-9fdc-1b88ce43caf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6827","nonce":2076704984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6829","nonce":2076704984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6833","nonce":2076704984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2076704984},{"type":"v1","addr":"172.21.15.79:6831","nonce":2076704984}]},"public_addr":"172.21.15.79:6827/2076704984","cluster_addr":"172.21.15.79:6829/2076704984","heartbeat_back_addr":"172.21.15.79:6833/2076704984","heartbeat_front_addr":"172.21.15.79:6831/2076704984","state":["exists","up"]},{"osd":2,"uuid":"0b1c9a57-cced-4163-a178-f461fc9596aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6819","nonce":3346893925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6821","nonce":3346893925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6825","nonce":3346893925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":3346893925},{"type":"v1","addr":"172.21.15.79:6823","nonce":3346893925}]},"public_addr":"172.21.15.79:6819/3346893925","cluster_addr":"172.21.15.79:6821/3346893925","heartbeat_back_addr":"172.21.15.79:6825/3346893925","heartbeat_front_addr":"172.21.15.79:6823/3346893925","state":["exists","up"]},{"osd":3,"uuid":"eab98798-a4c8-4e5f-83ec-40b3f31a4d50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6803","nonce":1649947840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6805","nonce":1649947840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6809","nonce":1649947840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":1649947840},{"type":"v1","addr":"172.21.15.79:6807","nonce":1649947840}]},"public_addr":"172.21.15.79:6803/1649947840","cluster_addr":"172.21.15.79:6805/1649947840","heartbeat_back_addr":"172.21.15.79:6809/1649947840","heartbeat_front_addr":"172.21.15.79:6807/1649947840","state":["exists","up"]},{"osd":4,"uuid":"6f17fdbb-5762-457a-8b89-4c5a4f2197b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6825","nonce":2788056220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6827","nonce":2788056220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6831","nonce":2788056220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2788056220},{"type":"v1","addr":"172.21.15.112:6829","nonce":2788056220}]},"public_addr":"172.21.15.112:6825/2788056220","cluster_addr":"172.21.15.112:6827/2788056220","heartbeat_back_addr":"172.21.15.112:6831/2788056220","heartbeat_front_addr":"172.21.15.112:6829/2788056220","state":["exists","up"]},{"osd":5,"uuid":"49ea8aae-4b05-43c9-979a-99026ae58a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6809","nonce":2352822260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6811","nonce":2352822260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6815","nonce":2352822260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":2352822260},{"type":"v1","addr":"172.21.15.112:6813","nonce":2352822260}]},"public_addr":"172.21.15.112:6809/2352822260","cluster_addr":"172.21.15.112:6811/2352822260","heartbeat_back_addr":"172.21.15.112:6815/2352822260","heartbeat_front_addr":"172.21.15.112:6813/2352822260","state":["exists","up"]},{"osd":6,"uuid":"ebb325d3-e1fc-44b6-929b-b7fdd71f82b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6801","nonce":4229386639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6803","nonce":4229386639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6807","nonce":4229386639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":4229386639},{"type":"v1","addr":"172.21.15.112:6805","nonce":4229386639}]},"public_addr":"172.21.15.112:6801/4229386639","cluster_addr":"172.21.15.112:6803/4229386639","heartbeat_back_addr":"172.21.15.112:6807/4229386639","heartbeat_front_addr":"172.21.15.112:6805/4229386639","state":["exists","up"]},{"osd":7,"uuid":"31813e1b-e740-49b2-9173-85911e32f3d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6817","nonce":3711700983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6819","nonce":3711700983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6823","nonce":3711700983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3711700983},{"type":"v1","addr":"172.21.15.112:6821","nonce":3711700983}]},"public_addr":"172.21.15.112:6817/3711700983","cluster_addr":"172.21.15.112:6819/3711700983","heartbeat_back_addr":"172.21.15.112:6823/3711700983","heartbeat_front_addr":"172.21.15.112:6821/3711700983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.906453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:10.017307+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:08.611770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.200083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.200292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.555977+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:07.001381+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-30T06:24:06.548682+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,0]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-30T06:31:52.898 INFO:tasks.ceph:Scrubbing osd.0 2023-10-30T06:31:52.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:53.090 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:53.091 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:53.091 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:53.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-30T06:31:53.648 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 0 to deep-scrub 2023-10-30T06:31:53.662 INFO:tasks.ceph:Scrubbing osd.1 2023-10-30T06:31:53.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:53.866 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:53.866 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:53.866 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:53.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-30T06:31:54.418 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 1 to deep-scrub 2023-10-30T06:31:54.431 INFO:tasks.ceph:Scrubbing osd.2 2023-10-30T06:31:54.432 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:54.622 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:54.622 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:54.623 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:54.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-30T06:31:55.262 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 2 to deep-scrub 2023-10-30T06:31:55.275 INFO:tasks.ceph:Scrubbing osd.3 2023-10-30T06:31:55.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:55.467 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:55.468 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:55.468 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:55.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-30T06:31:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 3 to deep-scrub 2023-10-30T06:31:56.051 INFO:tasks.ceph:Scrubbing osd.4 2023-10-30T06:31:56.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:56.268 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:56.269 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:56.269 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:56.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-30T06:31:56.819 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 4 to deep-scrub 2023-10-30T06:31:56.832 INFO:tasks.ceph:Scrubbing osd.5 2023-10-30T06:31:56.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:57.028 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:57.029 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:57.029 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:57.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-30T06:31:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 5 to deep-scrub 2023-10-30T06:31:57.662 INFO:tasks.ceph:Scrubbing osd.6 2023-10-30T06:31:57.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:57.862 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:57.863 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:57.863 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:57.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-30T06:31:58.433 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 6 to deep-scrub 2023-10-30T06:31:58.446 INFO:tasks.ceph:Scrubbing osd.7 2023-10-30T06:31:58.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-30T06:31:58.635 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-10-30T06:31:58.636 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-30T06:31:58.636 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-10-30T06:31:58.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-30T06:31:59.192 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 7 to deep-scrub 2023-10-30T06:31:59.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:31:59.813 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:31:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:31:59.826 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2023-10-30T06:31:59.335194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222068,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752652,"statfs":{"total":773094113280,"available":772866715648,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":67,"apply_latency_ms":67,"commit_latency_ns":67000000,"apply_latency_ns":67000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:41.056887+0000","last_change":"2023-10-30T06:31:21.233568+0000","last_active":"2023-10-30T06:31:41.056887+0000","last_peered":"2023-10-30T06:31:41.056887+0000","last_clean":"2023-10-30T06:31:41.056887+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:31:41.056887+0000","last_undegraded":"2023-10-30T06:31:41.056887+0000","last_fullsized":"2023-10-30T06:31:41.056887+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":144,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.059709+0000","last_change":"2023-10-30T06:31:21.236146+0000","last_active":"2023-10-30T06:31:40.059709+0000","last_peered":"2023-10-30T06:31:40.059709+0000","last_clean":"2023-10-30T06:31:40.059709+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:40.059709+0000","last_undegraded":"2023-10-30T06:31:40.059709+0000","last_fullsized":"2023-10-30T06:31:40.059709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T07:17:25.099081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.068274+0000","last_change":"2023-10-30T06:31:21.235163+0000","last_active":"2023-10-30T06:31:40.068274+0000","last_peered":"2023-10-30T06:31:40.068274+0000","last_clean":"2023-10-30T06:31:40.068274+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:31:40.068274+0000","last_undegraded":"2023-10-30T06:31:40.068274+0000","last_fullsized":"2023-10-30T06:31:40.068274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:45:06.831450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":148,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.060571+0000","last_change":"2023-10-30T06:28:34.905153+0000","last_active":"2023-10-30T06:31:40.060571+0000","last_peered":"2023-10-30T06:31:40.060571+0000","last_clean":"2023-10-30T06:31:40.060571+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:40.060571+0000","last_undegraded":"2023-10-30T06:31:40.060571+0000","last_fullsized":"2023-10-30T06:31:40.060571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T09:51:11.132065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:41.051269+0000","last_change":"2023-10-30T06:30:14.911176+0000","last_active":"2023-10-30T06:31:41.051269+0000","last_peered":"2023-10-30T06:31:41.051269+0000","last_clean":"2023-10-30T06:31:41.051269+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:31:41.051269+0000","last_undegraded":"2023-10-30T06:31:41.051269+0000","last_fullsized":"2023-10-30T06:31:41.051269+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":146,"state":"active+clean","last_fresh":"2023-10-30T06:31:40.065860+0000","last_change":"2023-10-30T06:24:18.948063+0000","last_active":"2023-10-30T06:31:40.065860+0000","last_peered":"2023-10-30T06:31:40.065860+0000","last_clean":"2023-10-30T06:31:40.065860+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:31:40.065860+0000","last_undegraded":"2023-10-30T06:31:40.065860+0000","last_fullsized":"2023-10-30T06:31:40.065860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T08:35:31.072238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:41.048477+0000","last_change":"2023-10-30T06:30:14.675186+0000","last_active":"2023-10-30T06:31:41.048477+0000","last_peered":"2023-10-30T06:31:41.048477+0000","last_clean":"2023-10-30T06:31:41.048477+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:31:41.048477+0000","last_undegraded":"2023-10-30T06:31:41.048477+0000","last_fullsized":"2023-10-30T06:31:41.048477+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:41.053915+0000","last_change":"2023-10-30T06:30:14.684655+0000","last_active":"2023-10-30T06:31:41.053915+0000","last_peered":"2023-10-30T06:31:41.053915+0000","last_clean":"2023-10-30T06:31:41.053915+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:31:41.053915+0000","last_undegraded":"2023-10-30T06:31:41.053915+0000","last_fullsized":"2023-10-30T06:31:41.053915+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:24:17.323069+0000","last_clean_scrub_stamp":"2023-10-30T06:24:17.323069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"147'78","reported_seq":249,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:55.370470+0000","last_change":"2023-10-30T06:31:55.382552+0000","last_active":"2023-10-30T06:31:55.370470+0000","last_peered":"2023-10-30T06:31:55.370470+0000","last_clean":"2023-10-30T06:31:55.370470+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:55.370470+0000","last_undegraded":"2023-10-30T06:31:55.370470+0000","last_fullsized":"2023-10-30T06:31:55.370470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'78","last_scrub_stamp":"2023-10-30T06:31:55.364578+0000","last_deep_scrub":"147'78","last_deep_scrub_stamp":"2023-10-30T06:31:55.364578+0000","last_clean_scrub_stamp":"2023-10-30T06:31:55.364578+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:41:36.755340+0000","scrub_duration":0.61865599100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411426,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":8.6620000000000008},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":18.204000000000001}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":7.4199999999999999},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":13.122}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":15.132},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":10.433999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":14.148999999999999},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":11.617000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":20.253},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":9.5570000000000004}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":21.085999999999999},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":12.231999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":17.452999999999999},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":18.885000000000002}]}]},{"osd":6,"up_from":16,"seq":68719476835,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":7.6719999999999997},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":14.282}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":14.657999999999999},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":13.561999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":6.9900000000000002},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":18.084}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":10.486000000000001},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":12.869}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":9.8130000000000006},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":19.061}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":15.393000000000001},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":12.193}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":19.887},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":16.981999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476835,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":13.351000000000001},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":6.718}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":9.2859999999999996},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":8.5589999999999993}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":11.677},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":16.184999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":15.221},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":7.391}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":9.8849999999999998},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":17.010000000000002}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":11.417999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":13.670999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":17.834},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":14.129}]}]},{"osd":2,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":12.361000000000001},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":9.4819999999999993}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":41.872},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":40.957000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":7.8780000000000001},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":14.904}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":10.773},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":8.2690000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":16.619},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":14.065}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":17.417999999999999},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":8.8729999999999993}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":15.529},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":11.510999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476834,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":7.766},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":10.253}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":8.1020000000000003},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":9.202}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":10.851000000000001},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":13.391}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":13.951000000000001},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":14.287000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":15.305},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":11.422000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":16.477},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":17.152999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":15.913},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":18.512}]}]},{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":25.666},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":26.763999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":7.0330000000000004},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":15.427}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":7.7210000000000001},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":8.4079999999999995}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":11.116},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":9.1720000000000006}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":10.535},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":14.430999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":12.311},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":11.67}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":16.227},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":13.667}]}]},{"osd":5,"up_from":15,"seq":64424509538,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":7.3200000000000003},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":4.3319999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":18.785},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":9.0980000000000008}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":12.744999999999999},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":10.757999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":13.483000000000001},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":9.9060000000000006}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":14.108000000000001},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":16.004999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":11.545999999999999},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":16.771999999999998}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":17.545000000000002},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":12.135999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509538,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":6.2000000000000002},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":9.1750000000000007}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":11.071999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":11.977}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":12.65},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":18.509}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":8.1630000000000003},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":20.399000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":13.397},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":14.75}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":15.382},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":17.760999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":19.475000000000001},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":14.007999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:31:59.831 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.831 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.832 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.833 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.833 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.833 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.834 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.834 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-30T06:24:17.323069+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=24, tm_sec=17, tm_wday=0, tm_yday=303, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=30, tm_hour=6, tm_min=31, tm_sec=51, tm_wday=0, tm_yday=303, tm_isdst=0) 2023-10-30T06:31:59.834 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-30T06:32:10.186 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:42.066 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.186 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:42.067 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.201 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:42.081 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.236 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:24.008 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.237 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:36.571 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.237 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:24.010 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.239 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:36.574 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.252 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:24.024 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:10.255 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:36.590 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:19.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-30T06:32:20.404 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-30T06:32:20.405 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-10-30T06:32:20.417 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2023-10-30T06:32:19.339297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222024,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752696,"statfs":{"total":773094113280,"available":772866760704,"internally_reserved":0,"allocated":6037504,"data_stored":3464736,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002830"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:03.092294+0000","last_change":"2023-10-30T06:32:03.100823+0000","last_active":"2023-10-30T06:32:03.092294+0000","last_peered":"2023-10-30T06:32:03.092294+0000","last_clean":"2023-10-30T06:32:03.092294+0000","last_became_active":"2023-10-30T06:31:21.227637+0000","last_became_peered":"2023-10-30T06:31:21.227637+0000","last_unstale":"2023-10-30T06:32:03.092294+0000","last_undegraded":"2023-10-30T06:32:03.092294+0000","last_fullsized":"2023-10-30T06:32:03.092294+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:03.088536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:03.088536+0000","last_clean_scrub_stamp":"2023-10-30T06:32:03.088536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:43:37.873675+0000","scrub_duration":0.21491264299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:57.997508+0000","last_change":"2023-10-30T06:31:58.006329+0000","last_active":"2023-10-30T06:31:57.997508+0000","last_peered":"2023-10-30T06:31:57.997508+0000","last_clean":"2023-10-30T06:31:57.997508+0000","last_became_active":"2023-10-30T06:31:21.229960+0000","last_became_peered":"2023-10-30T06:31:21.229960+0000","last_unstale":"2023-10-30T06:31:57.997508+0000","last_undegraded":"2023-10-30T06:31:57.997508+0000","last_fullsized":"2023-10-30T06:31:57.997508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:31:57.993627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:31:57.993627+0000","last_clean_scrub_stamp":"2023-10-30T06:31:57.993627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T17:25:28.235503+0000","scrub_duration":0.36423344000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":34,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:13.407137+0000","last_change":"2023-10-30T06:32:13.408167+0000","last_active":"2023-10-30T06:32:13.407137+0000","last_peered":"2023-10-30T06:32:13.407137+0000","last_clean":"2023-10-30T06:32:13.407137+0000","last_became_active":"2023-10-30T06:31:21.228709+0000","last_became_peered":"2023-10-30T06:31:21.228709+0000","last_unstale":"2023-10-30T06:32:13.407137+0000","last_undegraded":"2023-10-30T06:32:13.407137+0000","last_fullsized":"2023-10-30T06:32:13.407137+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:13.405795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:13.405795+0000","last_clean_scrub_stamp":"2023-10-30T06:32:13.405795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T16:57:06.222586+0000","scrub_duration":0.031788716000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:58.189683+0000","last_change":"2023-10-30T06:31:58.198011+0000","last_active":"2023-10-30T06:31:58.189683+0000","last_peered":"2023-10-30T06:31:58.189683+0000","last_clean":"2023-10-30T06:31:58.189683+0000","last_became_active":"2023-10-30T06:28:34.898980+0000","last_became_peered":"2023-10-30T06:28:34.898980+0000","last_unstale":"2023-10-30T06:31:58.189683+0000","last_undegraded":"2023-10-30T06:31:58.189683+0000","last_fullsized":"2023-10-30T06:31:58.189683+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:31:58.184906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:31:58.184906+0000","last_clean_scrub_stamp":"2023-10-30T06:31:58.184906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T14:46:23.668571+0000","scrub_duration":0.29904202200000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":59,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:08.859284+0000","last_change":"2023-10-30T06:32:08.863783+0000","last_active":"2023-10-30T06:32:08.859284+0000","last_peered":"2023-10-30T06:32:08.859284+0000","last_clean":"2023-10-30T06:32:08.859284+0000","last_became_active":"2023-10-30T06:30:14.905638+0000","last_became_peered":"2023-10-30T06:30:14.905638+0000","last_unstale":"2023-10-30T06:32:08.859284+0000","last_undegraded":"2023-10-30T06:32:08.859284+0000","last_fullsized":"2023-10-30T06:32:08.859284+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:08.857950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:08.857950+0000","last_clean_scrub_stamp":"2023-10-30T06:32:08.857950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T10:51:30.508996+0000","scrub_duration":0.029054066,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:12.562227+0000","last_change":"2023-10-30T06:32:12.571453+0000","last_active":"2023-10-30T06:32:12.562227+0000","last_peered":"2023-10-30T06:32:12.562227+0000","last_clean":"2023-10-30T06:32:12.562227+0000","last_became_active":"2023-10-30T06:24:18.916760+0000","last_became_peered":"2023-10-30T06:24:18.916760+0000","last_unstale":"2023-10-30T06:32:12.562227+0000","last_undegraded":"2023-10-30T06:32:12.562227+0000","last_fullsized":"2023-10-30T06:32:12.562227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:12.555536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:12.555536+0000","last_clean_scrub_stamp":"2023-10-30T06:32:12.555536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:42:24.461541+0000","scrub_duration":0.21768158500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:06.894249+0000","last_change":"2023-10-30T06:32:06.894249+0000","last_active":"2023-10-30T06:32:06.894249+0000","last_peered":"2023-10-30T06:32:06.894249+0000","last_clean":"2023-10-30T06:32:06.894249+0000","last_became_active":"2023-10-30T06:30:14.669286+0000","last_became_peered":"2023-10-30T06:30:14.669286+0000","last_unstale":"2023-10-30T06:32:06.894249+0000","last_undegraded":"2023-10-30T06:32:06.894249+0000","last_fullsized":"2023-10-30T06:32:06.894249+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:06.891861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:06.891861+0000","last_clean_scrub_stamp":"2023-10-30T06:32:06.891861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T13:10:56.846203+0000","scrub_duration":0.031537857000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":59,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:32:05.033935+0000","last_change":"2023-10-30T06:32:05.038388+0000","last_active":"2023-10-30T06:32:05.033935+0000","last_peered":"2023-10-30T06:32:05.033935+0000","last_clean":"2023-10-30T06:32:05.033935+0000","last_became_active":"2023-10-30T06:30:14.678916+0000","last_became_peered":"2023-10-30T06:30:14.678916+0000","last_unstale":"2023-10-30T06:32:05.033935+0000","last_undegraded":"2023-10-30T06:32:05.033935+0000","last_fullsized":"2023-10-30T06:32:05.033935+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-30T06:32:05.032643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-30T06:32:05.032643+0000","last_clean_scrub_stamp":"2023-10-30T06:32:05.032643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T15:58:47.997752+0000","scrub_duration":0.135963163,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"147'78","reported_seq":250,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-30T06:31:55.382552+0000","last_change":"2023-10-30T06:31:55.382552+0000","last_active":"2023-10-30T06:31:55.382552+0000","last_peered":"2023-10-30T06:31:55.382552+0000","last_clean":"2023-10-30T06:31:55.382552+0000","last_became_active":"2023-10-30T06:24:14.625588+0000","last_became_peered":"2023-10-30T06:24:14.625588+0000","last_unstale":"2023-10-30T06:31:55.382552+0000","last_undegraded":"2023-10-30T06:31:55.382552+0000","last_fullsized":"2023-10-30T06:31:55.382552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'78","last_scrub_stamp":"2023-10-30T06:31:55.364578+0000","last_deep_scrub":"147'78","last_deep_scrub_stamp":"2023-10-30T06:31:55.364578+0000","last_clean_scrub_stamp":"2023-10-30T06:31:55.364578+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-31T12:41:36.755340+0000","scrub_duration":0.61865599100000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":795680,"data_compressed":1378,"data_compressed_allocated":49152,"data_compressed_original":778240,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":18,"seq":77309411430,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":11.295,"5min":10.743,"15min":10.532},"min":{"1min":5.5869999999999997,"5min":4.6680000000000001,"15min":4.6680000000000001},"max":{"1min":31.196000000000002,"5min":51.225000000000001,"15min":51.225000000000001},"last":10.409000000000001},{"interface":"front","average":{"1min":10.276999999999999,"5min":10.615,"15min":11.596},"min":{"1min":6.351,"5min":5.1260000000000003,"15min":5.1260000000000003},"max":{"1min":18.286999999999999,"5min":54.011000000000003,"15min":54.011000000000003},"last":8.8290000000000006}]},{"osd":2,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":9.8149999999999995,"5min":10.926,"15min":12.788},"min":{"1min":6.2869999999999999,"5min":5.3940000000000001,"15min":5.3940000000000001},"max":{"1min":20.823,"5min":31.373000000000001,"15min":52.552},"last":12.557},{"interface":"front","average":{"1min":14.452999999999999,"5min":14.259,"15min":14.984},"min":{"1min":7,"5min":7,"15min":7},"max":{"1min":24.806000000000001,"5min":40.183,"15min":55.585999999999999},"last":7.1820000000000004}]},{"osd":3,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.318999999999999,"5min":16.873000000000001,"15min":20.562999999999999},"min":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"max":{"1min":30.547000000000001,"5min":48.716000000000001,"15min":118.45699999999999},"last":11.205},{"interface":"front","average":{"1min":12.912000000000001,"5min":14.234999999999999,"15min":19.277000000000001},"min":{"1min":6.9720000000000004,"5min":6.9720000000000004,"15min":6.9720000000000004},"max":{"1min":25.637,"5min":50.332999999999998,"15min":124.075},"last":7.532}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":21.402999999999999,"5min":15.404,"15min":14.275},"min":{"1min":11.084,"5min":6.9809999999999999,"15min":6.9809999999999999},"max":{"1min":81.524000000000001,"5min":81.524000000000001,"15min":81.524000000000001},"last":13.249000000000001},{"interface":"front","average":{"1min":17.271000000000001,"5min":16.995000000000001,"15min":17.68},"min":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"max":{"1min":82.909000000000006,"5min":82.909000000000006,"15min":82.909000000000006},"last":9.468}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.724,"5min":16.294,"15min":19.166},"min":{"1min":10.419,"5min":8.0500000000000007,"15min":7.351},"max":{"1min":36.783000000000001,"5min":36.783000000000001,"15min":123.754},"last":13.875999999999999},{"interface":"front","average":{"1min":18.452000000000002,"5min":16.335000000000001,"15min":20.271000000000001},"min":{"1min":8.7810000000000006,"5min":6.8520000000000003,"15min":6.8520000000000003},"max":{"1min":35.073,"5min":35.601999999999997,"15min":122.72499999999999},"last":11.928000000000001}]},{"osd":6,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":17.428999999999998,"5min":17.428999999999998,"15min":19.882000000000001},"min":{"1min":9.641,"5min":9.3279999999999994,"15min":9.3279999999999994},"max":{"1min":31.658000000000001,"5min":37.066000000000003,"15min":66.400999999999996},"last":14.561999999999999},{"interface":"front","average":{"1min":19.068000000000001,"5min":17.846,"15min":18.986000000000001},"min":{"1min":8.7989999999999995,"5min":8.1929999999999996,"15min":8.1929999999999996},"max":{"1min":37.009999999999998,"5min":43.159999999999997,"15min":53.149000000000001},"last":17.245000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":19.809999999999999,"5min":19.324000000000002,"15min":20.581},"min":{"1min":11.811999999999999,"5min":8.8360000000000003,"15min":8.8360000000000003},"max":{"1min":36.987000000000002,"5min":48.520000000000003,"15min":51.375},"last":18.242000000000001},{"interface":"front","average":{"1min":18.094999999999999,"5min":18.986999999999998,"15min":20.605},"min":{"1min":11.792,"5min":8.6739999999999995,"15min":8.6739999999999995},"max":{"1min":27.242000000000001,"5min":47.256999999999998,"15min":69.991},"last":15.201000000000001}]}]},{"osd":6,"up_from":16,"seq":68719476839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":9.5570000000000004,"5min":10.276,"15min":12.952999999999999},"min":{"1min":4.4409999999999998,"5min":4.4409999999999998,"15min":4.4409999999999998},"max":{"1min":21.745999999999999,"5min":31.138000000000002,"15min":94.343000000000004},"last":5.6200000000000001},{"interface":"front","average":{"1min":8.484,"5min":9.6240000000000006,"15min":13.456},"min":{"1min":4.2839999999999998,"5min":4.2839999999999998,"15min":4.2839999999999998},"max":{"1min":20.488,"5min":22.806000000000001,"15min":122.054},"last":8.8629999999999995}]},{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.289999999999999,"5min":12.156000000000001,"15min":11.212999999999999},"min":{"1min":7.3200000000000003,"5min":5.1299999999999999,"15min":5.1299999999999999},"max":{"1min":53.445,"5min":53.445,"15min":53.445},"last":10.565},{"interface":"front","average":{"1min":17.071000000000002,"5min":14.708,"15min":26.105},"min":{"1min":9.4990000000000006,"5min":5.9989999999999997,"15min":5.9269999999999996},"max":{"1min":52.515000000000001,"5min":52.515000000000001,"15min":387.15600000000001},"last":12.352}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.076000000000001,"5min":13.499000000000001,"15min":17.277999999999999},"min":{"1min":6.694,"5min":6.5999999999999996,"15min":6.4359999999999999},"max":{"1min":23.053999999999998,"5min":72.988,"15min":156.90299999999999},"last":11.750999999999999},{"interface":"front","average":{"1min":17.428000000000001,"5min":17.292999999999999,"15min":19.064},"min":{"1min":10.207000000000001,"5min":6.1239999999999997,"15min":6.1239999999999997},"max":{"1min":30.224,"5min":75.004000000000005,"15min":143.578},"last":16.472000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":14.209,"5min":16.417999999999999,"15min":19.658000000000001},"min":{"1min":9.2850000000000001,"5min":7.7999999999999998,"15min":7.7999999999999998},"max":{"1min":25.442,"5min":50.095999999999997,"15min":141.18799999999999},"last":13.000999999999999},{"interface":"front","average":{"1min":19.792999999999999,"5min":19.52,"15min":22.983000000000001},"min":{"1min":11.800000000000001,"5min":9.1099999999999994,"15min":9.1099999999999994},"max":{"1min":39.451000000000001,"5min":50.887999999999998,"15min":161.32900000000001},"last":13.686}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":19.007999999999999,"5min":14.297000000000001,"15min":14.531000000000001},"min":{"1min":11.635,"5min":7.282,"15min":6.2240000000000002},"max":{"1min":33.350000000000001,"5min":33.350000000000001,"15min":74.079999999999998},"last":17.158999999999999},{"interface":"front","average":{"1min":12.348000000000001,"5min":14.148,"15min":15.042},"min":{"1min":7.157,"5min":6.782,"15min":6.782},"max":{"1min":29.821000000000002,"5min":29.821000000000002,"15min":58.613999999999997},"last":10.257}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.789,"5min":14.042,"15min":16.379999999999999},"min":{"1min":7.3419999999999996,"5min":7.3419999999999996,"15min":7.3419999999999996},"max":{"1min":22.396000000000001,"5min":48.072000000000003,"15min":71.757000000000005},"last":18.228000000000002},{"interface":"front","average":{"1min":19.187000000000001,"5min":19.433,"15min":20.725999999999999},"min":{"1min":10.984,"5min":9.532,"15min":9.532},"max":{"1min":36.634999999999998,"5min":49.463999999999999,"15min":80.119},"last":15.997}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":16.606999999999999,"5min":15.752000000000001,"15min":18.183},"min":{"1min":10.545,"5min":9.1289999999999996,"15min":8.9610000000000003},"max":{"1min":27.902999999999999,"5min":27.902999999999999,"15min":81.247},"last":19.771999999999998},{"interface":"front","average":{"1min":19.776,"5min":17.811,"15min":18.466999999999999},"min":{"1min":7.4779999999999998,"5min":7.4779999999999998,"15min":7.4779999999999998},"max":{"1min":34.741999999999997,"5min":34.741999999999997,"15min":70.989000000000004},"last":17.786999999999999}]}]},{"osd":3,"up_from":16,"seq":68719476839,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":8.9280000000000008,"5min":10.087999999999999,"15min":10.676},"min":{"1min":4.9359999999999999,"5min":4.423,"15min":4.2679999999999998},"max":{"1min":14.516,"5min":48.792000000000002,"15min":48.792000000000002},"last":4.4000000000000004},{"interface":"front","average":{"1min":10.215,"5min":10.744999999999999,"15min":10.486000000000001},"min":{"1min":4.8399999999999999,"5min":4.8399999999999999,"15min":4.7220000000000004},"max":{"1min":19.597999999999999,"5min":28.616,"15min":28.616},"last":12.343999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.172999999999998,"5min":15.071,"15min":21.965},"min":{"1min":7.9619999999999997,"5min":6.2939999999999996,"15min":6.2939999999999996},"max":{"1min":72.522000000000006,"5min":81.552999999999997,"15min":165.76400000000001},"last":13.930999999999999},{"interface":"front","average":{"1min":18.108000000000001,"5min":14.84,"15min":22.655000000000001},"min":{"1min":7.5919999999999996,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":71.013999999999996,"5min":82.995000000000005,"15min":190.34200000000001},"last":13.132999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.265000000000001,"5min":15.57,"15min":17.303999999999998},"min":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"max":{"1min":30.076000000000001,"5min":42.881999999999998,"15min":42.881999999999998},"last":14.68},{"interface":"front","average":{"1min":14.318,"5min":15.773,"15min":16.526},"min":{"1min":8.9969999999999999,"5min":8.0449999999999999,"15min":8.0449999999999999},"max":{"1min":27.059000000000001,"5min":71.721000000000004,"15min":71.721000000000004},"last":8.8059999999999992}]},{"osd":4,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":18.614999999999998,"5min":17.838999999999999,"15min":19.346},"min":{"1min":9.3780000000000001,"5min":6.6040000000000001,"15min":6.6040000000000001},"max":{"1min":42.607999999999997,"5min":44.091000000000001,"15min":44.091000000000001},"last":16.359999999999999},{"interface":"front","average":{"1min":11.891999999999999,"5min":11.507999999999999,"15min":11.456},"min":{"1min":7.6109999999999998,"5min":5.9400000000000004,"15min":5.7969999999999997},"max":{"1min":29.103000000000002,"5min":45.231000000000002,"15min":45.231000000000002},"last":7.8730000000000002}]},{"osd":5,"last update":"Mon Oct 30 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":15.381,"5min":15.632,"15min":17.873000000000001},"min":{"1min":6.9340000000000002,"5min":6.0069999999999997,"15min":6.0069999999999997},"max":{"1min":59.384999999999998,"5min":59.384999999999998,"15min":59.384999999999998},"last":9.5579999999999998},{"interface":"front","average":{"1min":15.567,"5min":15.092000000000001,"15min":18.571999999999999},"min":{"1min":9.5920000000000005,"5min":8.0399999999999991,"15min":8.0399999999999991},"max":{"1min":25.295000000000002,"5min":28.882000000000001,"15min":58.576000000000001},"last":17.684999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":15.231999999999999,"5min":15.48,"15min":23.943999999999999},"min":{"1min":7.5519999999999996,"5min":7.0469999999999997,"15min":7.0469999999999997},"max":{"1min":28.030999999999999,"5min":29.905000000000001,"15min":169.05199999999999},"last":10.545999999999999},{"interface":"front","average":{"1min":17.547000000000001,"5min":16.120999999999999,"15min":15.653},"min":{"1min":9.2989999999999995,"5min":6.5510000000000002,"15min":6.5510000000000002},"max":{"1min":30.756,"5min":34.969000000000001,"15min":40.128999999999998},"last":11.223000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:24 2023","interfaces":[{"interface":"back","average":{"1min":16.263999999999999,"5min":19.334,"15min":18.361000000000001},"min":{"1min":11.260999999999999,"5min":9.5519999999999996,"15min":8.5950000000000006},"max":{"1min":23.978999999999999,"5min":64.313999999999993,"15min":64.313999999999993},"last":15.768000000000001},{"interface":"front","average":{"1min":20.559000000000001,"5min":20.286999999999999,"15min":25.821999999999999},"min":{"1min":10.984,"5min":9.5990000000000002,"15min":9.5990000000000002},"max":{"1min":47.518999999999998,"5min":55.331000000000003,"15min":170.232},"last":16.989999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:19 2023","interfaces":[{"interface":"back","average":{"1min":12.898999999999999,"5min":11.378,"15min":21.315000000000001},"min":{"1min":5.3819999999999997,"5min":3.8090000000000002,"15min":3.8090000000000002},"max":{"1min":35.482999999999997,"5min":91.290999999999997,"15min":258.66199999999998},"last":17.305},{"interface":"front","average":{"1min":12.340999999999999,"5min":11.583,"15min":18.963000000000001},"min":{"1min":6.7930000000000001,"5min":5.3499999999999996,"15min":5.3499999999999996},"max":{"1min":36.673999999999999,"5min":92.879999999999995,"15min":203.38499999999999},"last":8.0579999999999998}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.504,"5min":12.538,"15min":14.782999999999999},"min":{"1min":8.093,"5min":5.3419999999999996,"15min":5.3419999999999996},"max":{"1min":27.315000000000001,"5min":31.689,"15min":45.936999999999998},"last":19.373999999999999},{"interface":"front","average":{"1min":14.279,"5min":13.388,"15min":14.137},"min":{"1min":6.407,"5min":5.7140000000000004,"15min":5.7140000000000004},"max":{"1min":31.271000000000001,"5min":35.034999999999997,"15min":37.390999999999998},"last":9.6790000000000003}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":12.253,"5min":11.034000000000001,"15min":15.978},"min":{"1min":6.1210000000000004,"5min":6.1210000000000004,"15min":6.1210000000000004},"max":{"1min":27.789000000000001,"5min":27.789000000000001,"15min":183.38900000000001},"last":21.381},{"interface":"front","average":{"1min":18.173999999999999,"5min":15.723000000000001,"15min":20.021999999999998},"min":{"1min":9.1120000000000001,"5min":7.6399999999999997,"15min":6.8810000000000002},"max":{"1min":31.774999999999999,"5min":31.774999999999999,"15min":189.374},"last":20.048999999999999}]},{"osd":4,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.888,"5min":13.712,"15min":18.751000000000001},"min":{"1min":6.8380000000000001,"5min":6.8380000000000001,"15min":6.8380000000000001},"max":{"1min":35.671999999999997,"5min":35.671999999999997,"15min":186.01400000000001},"last":22.026},{"interface":"front","average":{"1min":16.164000000000001,"5min":15.369,"15min":18.053000000000001},"min":{"1min":7.9900000000000002,"5min":6.7679999999999998,"15min":6.2400000000000002},"max":{"1min":33.32,"5min":36.784999999999997,"15min":167.67599999999999},"last":17.989000000000001}]},{"osd":5,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.635000000000002,"5min":18.126999999999999,"15min":23.667000000000002},"min":{"1min":9.3040000000000003,"5min":8.843,"15min":8.3040000000000003},"max":{"1min":34.206000000000003,"5min":65.251999999999995,"15min":215.80500000000001},"last":24.800000000000001},{"interface":"front","average":{"1min":16.725999999999999,"5min":16.960999999999999,"15min":23.087},"min":{"1min":9.7119999999999997,"5min":8.8520000000000003,"15min":8.6989999999999998},"max":{"1min":26.161999999999999,"5min":34.042000000000002,"15min":187.863},"last":18.712}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":20.591000000000001,"5min":18.273,"15min":23.472000000000001},"min":{"1min":10.499000000000001,"5min":7.7969999999999997,"15min":7.7969999999999997},"max":{"1min":36.698,"5min":66.673000000000002,"15min":228.81200000000001},"last":20.751000000000001},{"interface":"front","average":{"1min":15.128,"5min":18.158000000000001,"15min":25.507999999999999},"min":{"1min":7.766,"5min":7.766,"15min":7.766},"max":{"1min":31.981000000000002,"5min":45.685000000000002,"15min":275.92200000000003},"last":27.475000000000001}]},{"osd":7,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":18.989000000000001,"5min":19.006,"15min":23.849},"min":{"1min":12.554,"5min":9.4580000000000002,"15min":9.4580000000000002},"max":{"1min":28.102,"5min":35.898000000000003,"15min":170.43600000000001},"last":25.742999999999999},{"interface":"front","average":{"1min":16.933,"5min":17.465,"15min":21.077999999999999},"min":{"1min":9.5909999999999993,"5min":8.5280000000000005,"15min":8.5280000000000005},"max":{"1min":30.628,"5min":32.140000000000001,"15min":157.00200000000001},"last":26.681999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":8.8949999999999996,"5min":9.9489999999999998,"15min":10.535},"min":{"1min":5.1180000000000003,"5min":4.601,"15min":4.5149999999999997},"max":{"1min":17.07,"5min":56.704000000000001,"15min":56.704000000000001},"last":5.5590000000000002},{"interface":"front","average":{"1min":9.1310000000000002,"5min":10.869,"15min":11.515000000000001},"min":{"1min":5.3159999999999998,"5min":4.5949999999999998,"15min":4.5949999999999998},"max":{"1min":25.234999999999999,"5min":57.805,"15min":57.805},"last":12.587}]},{"osd":2,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":11.916,"5min":10.637,"15min":16.273},"min":{"1min":6.4660000000000002,"5min":5.6660000000000004,"15min":5.6660000000000004},"max":{"1min":24.372,"5min":32.957000000000001,"15min":127.95399999999999},"last":11.795},{"interface":"front","average":{"1min":13.436,"5min":12.587999999999999,"15min":13.948},"min":{"1min":7.9059999999999997,"5min":6.6929999999999996,"15min":6.6929999999999996},"max":{"1min":29.853999999999999,"5min":35.310000000000002,"15min":35.834000000000003},"last":7.2290000000000001}]},{"osd":3,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.902000000000001,"5min":14.773,"15min":19.468},"min":{"1min":10.298999999999999,"5min":7.7949999999999999,"15min":6.9530000000000003},"max":{"1min":47.862000000000002,"5min":47.862000000000002,"15min":99.790999999999997},"last":13.257},{"interface":"front","average":{"1min":16.626999999999999,"5min":16.129999999999999,"15min":16.367999999999999},"min":{"1min":8.1910000000000007,"5min":6.516,"15min":6.516},"max":{"1min":45.436999999999998,"5min":45.436999999999998,"15min":45.436999999999998},"last":9.4749999999999996}]},{"osd":4,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":13.715999999999999,"5min":13.651,"15min":15.302},"min":{"1min":8.3089999999999993,"5min":7.1559999999999997,"15min":6.8150000000000004},"max":{"1min":28.521000000000001,"5min":51.802,"15min":51.802},"last":13.984},{"interface":"front","average":{"1min":19.151,"5min":17.042000000000002,"15min":21.914999999999999},"min":{"1min":7.4589999999999996,"5min":7.4589999999999996,"15min":7.4589999999999996},"max":{"1min":31.611999999999998,"5min":40.000999999999998,"15min":103.705},"last":15.260999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":15.807,"5min":14.955,"15min":17.350000000000001},"min":{"1min":8.6150000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"max":{"1min":32.5,"5min":42.204999999999998,"15min":43.847000000000001},"last":14.627000000000001},{"interface":"front","average":{"1min":19.940000000000001,"5min":18.623999999999999,"15min":24.347999999999999},"min":{"1min":9.9640000000000004,"5min":8.8000000000000007,"15min":8.8000000000000007},"max":{"1min":33.906999999999996,"5min":40.598999999999997,"15min":98.707999999999998},"last":18.556999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":17.076000000000001,"5min":17.318999999999999,"15min":19.302},"min":{"1min":9.2789999999999999,"5min":9.2789999999999999,"15min":8.6869999999999994},"max":{"1min":32.875999999999998,"5min":51.018999999999998,"15min":55.631},"last":8.0139999999999993},{"interface":"front","average":{"1min":18.088000000000001,"5min":17.065999999999999,"15min":20.265999999999998},"min":{"1min":10.358000000000001,"5min":8.2080000000000002,"15min":8.2080000000000002},"max":{"1min":35.758000000000003,"5min":48.860999999999997,"15min":54.656999999999996},"last":15.891}]},{"osd":7,"last update":"Mon Oct 30 06:31:23 2023","interfaces":[{"interface":"back","average":{"1min":20.437999999999999,"5min":19.298999999999999,"15min":20.844000000000001},"min":{"1min":10.432,"5min":9.8219999999999992,"15min":9.8219999999999992},"max":{"1min":37.826000000000001,"5min":53.314,"15min":53.314},"last":12.176},{"interface":"front","average":{"1min":21.157,"5min":19.145,"15min":21.518000000000001},"min":{"1min":12.242000000000001,"5min":10.534000000000001,"15min":10.534000000000001},"max":{"1min":37.652000000000001,"5min":47.578000000000003,"15min":47.578000000000003},"last":17.786000000000001}]}]},{"osd":7,"up_from":15,"seq":64424509542,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":9.5510000000000002,"5min":10.971,"15min":14.066000000000001},"min":{"1min":4.5860000000000003,"5min":4.4470000000000001,"15min":4.4249999999999998},"max":{"1min":26.701000000000001,"5min":109.586,"15min":131.477},"last":8.8010000000000002},{"interface":"front","average":{"1min":12.039999999999999,"5min":12.25,"15min":14.625999999999999},"min":{"1min":6.1440000000000001,"5min":5.8920000000000003,"15min":5.6340000000000003},"max":{"1min":27.597000000000001,"5min":89.219999999999999,"15min":129.386},"last":10.875999999999999}]},{"osd":1,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":12.85,"5min":12.753,"15min":11.534000000000001},"min":{"1min":7.1779999999999999,"5min":6.0599999999999996,"15min":6.0599999999999996},"max":{"1min":24.809000000000001,"5min":68.385000000000005,"15min":68.385000000000005},"last":14.372999999999999},{"interface":"front","average":{"1min":19.402999999999999,"5min":18.016999999999999,"15min":18.777999999999999},"min":{"1min":10.029,"5min":7.4429999999999996,"15min":7.4429999999999996},"max":{"1min":30.509,"5min":77.412999999999997,"15min":91.295000000000002},"last":15.436}]},{"osd":2,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":13.866,"5min":12.271000000000001,"15min":15.880000000000001},"min":{"1min":7.423,"5min":5.1470000000000002,"15min":5.1470000000000002},"max":{"1min":28.661999999999999,"5min":31.763000000000002,"15min":134.21199999999999},"last":5.9580000000000002},{"interface":"front","average":{"1min":16.356999999999999,"5min":13.667,"15min":14.311999999999999},"min":{"1min":6.3170000000000002,"5min":6.0830000000000002,"15min":5.992},"max":{"1min":31.791,"5min":33.060000000000002,"15min":50.450000000000003},"last":10.481999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":18.954999999999998,"5min":15.882,"15min":18.553999999999998},"min":{"1min":6.1669999999999998,"5min":6.1669999999999998,"15min":6.1669999999999998},"max":{"1min":62.503,"5min":67.840000000000003,"15min":132.56100000000001},"last":12.994},{"interface":"front","average":{"1min":22.719000000000001,"5min":17.754000000000001,"15min":19.684000000000001},"min":{"1min":11.939,"5min":10.116,"15min":6.5780000000000003},"max":{"1min":61.453000000000003,"5min":66.106999999999999,"15min":124.684},"last":16.227}]},{"osd":4,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":16.079000000000001,"5min":14.843,"15min":14.119999999999999},"min":{"1min":8.2690000000000001,"5min":6.2409999999999997,"15min":6.2409999999999997},"max":{"1min":34.628999999999998,"5min":36.552,"15min":36.552},"last":11.837},{"interface":"front","average":{"1min":18.640999999999998,"5min":17.010000000000002,"15min":16.111999999999998},"min":{"1min":11.433999999999999,"5min":9.9429999999999996,"15min":8.3089999999999993},"max":{"1min":27.838999999999999,"5min":36.908999999999999,"15min":36.908999999999999},"last":16.856999999999999}]},{"osd":5,"last update":"Mon Oct 30 06:31:28 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.996,"15min":17.635999999999999},"min":{"1min":7.2610000000000001,"5min":6.6790000000000003,"15min":6.6790000000000003},"max":{"1min":33.814,"5min":33.814,"15min":125.792},"last":7.0970000000000004},{"interface":"front","average":{"1min":18.821999999999999,"5min":17.262,"15min":17.058},"min":{"1min":10.313000000000001,"5min":8.3599999999999994,"15min":8.3599999999999994},"max":{"1min":35.694000000000003,"5min":35.694000000000003,"15min":35.694000000000003},"last":13.677}]},{"osd":6,"last update":"Mon Oct 30 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":20.815999999999999,"5min":18.864000000000001,"15min":26.667999999999999},"min":{"1min":10.782999999999999,"5min":9.4559999999999995,"15min":7.6020000000000003},"max":{"1min":32.765999999999998,"5min":59.167000000000002,"15min":327.29599999999999},"last":17.997},{"interface":"front","average":{"1min":14.529,"5min":15.037000000000001,"15min":26.321000000000002},"min":{"1min":8.3629999999999995,"5min":8.1289999999999996,"15min":7.5499999999999998},"max":{"1min":22.539000000000001,"5min":57.106999999999999,"15min":323.82400000000001},"last":8.2439999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":782336,"data_stored":731472,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":8.6940000000000008,"5min":10.167,"15min":13.148},"min":{"1min":4.556,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":15.476000000000001,"5min":51.149000000000001,"15min":68.513999999999996},"last":6.3449999999999998},{"interface":"front","average":{"1min":10.846,"5min":11.063000000000001,"15min":14.631},"min":{"1min":4.8650000000000002,"5min":4.8650000000000002,"15min":4.8650000000000002},"max":{"1min":18.646999999999998,"5min":50.776000000000003,"15min":88.816000000000003},"last":5}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.050000000000001,"5min":14.443,"15min":16.190999999999999},"min":{"1min":7.0549999999999997,"5min":6.2869999999999999,"15min":6.2869999999999999},"max":{"1min":33.710999999999999,"5min":63.587000000000003,"15min":87.515000000000001},"last":13.984},{"interface":"front","average":{"1min":17.350000000000001,"5min":15.629,"15min":16.859999999999999},"min":{"1min":8.234,"5min":6.8099999999999996,"15min":6.21},"max":{"1min":46.723999999999997,"5min":65.349000000000004,"15min":85.832999999999998},"last":10.872}]},{"osd":2,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":16.350999999999999,"5min":15.371,"15min":17.141999999999999},"min":{"1min":8.0419999999999998,"5min":6.5259999999999998,"15min":6.5259999999999998},"max":{"1min":29.512,"5min":36.207000000000001,"15min":113.943},"last":11.875999999999999},{"interface":"front","average":{"1min":13.145,"5min":11.416,"15min":13.826000000000001},"min":{"1min":8.1010000000000009,"5min":5.6589999999999998,"15min":5.6589999999999998},"max":{"1min":27.565999999999999,"5min":33.826000000000001,"15min":80.004000000000005},"last":9.1880000000000006}]},{"osd":3,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":19.068999999999999,"5min":16.189,"15min":19.600999999999999},"min":{"1min":11.108000000000001,"5min":6.4770000000000003,"15min":6.2889999999999997},"max":{"1min":30.885999999999999,"5min":36.628,"15min":87.866},"last":13.34},{"interface":"front","average":{"1min":13.272,"5min":15.563000000000001,"15min":20.859000000000002},"min":{"1min":6.8970000000000002,"5min":6.8970000000000002,"15min":6.8970000000000002},"max":{"1min":22.742000000000001,"5min":29.518999999999998,"15min":91.183000000000007},"last":16.120000000000001}]},{"osd":4,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":20.015999999999998,"5min":15.773,"15min":13.749000000000001},"min":{"1min":14.337999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"max":{"1min":32.104999999999997,"5min":41.762999999999998,"15min":41.762999999999998},"last":12.689},{"interface":"front","average":{"1min":14.51,"5min":15.631,"15min":16.460000000000001},"min":{"1min":6.9870000000000001,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":24.831,"5min":42.552,"15min":59.609000000000002},"last":9.8309999999999995}]},{"osd":6,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":15.753,"5min":15.640000000000001,"15min":15.829000000000001},"min":{"1min":9.5340000000000007,"5min":7.9820000000000002,"15min":7.6980000000000004},"max":{"1min":28.094000000000001,"5min":45.054000000000002,"15min":66.765000000000001},"last":17.966999999999999},{"interface":"front","average":{"1min":14.417,"5min":17.655999999999999,"15min":19.832999999999998},"min":{"1min":8.9380000000000006,"5min":6.6079999999999997,"15min":6.6079999999999997},"max":{"1min":20.742999999999999,"5min":45.877000000000002,"15min":71.771000000000001},"last":14.731999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:18 2023","interfaces":[{"interface":"back","average":{"1min":21.053000000000001,"5min":18.497,"15min":18.788},"min":{"1min":11.840999999999999,"5min":8.3759999999999994,"15min":8.3759999999999994},"max":{"1min":34.463000000000001,"5min":35.192,"15min":64.677000000000007},"last":15.382999999999999},{"interface":"front","average":{"1min":18.529,"5min":16.751999999999999,"15min":17.657},"min":{"1min":11.153,"5min":7.7110000000000003,"15min":7.7110000000000003},"max":{"1min":26.271999999999998,"5min":29.597999999999999,"15min":42.531999999999996},"last":18.829000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":745472,"data_stored":333632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.6500000000000004,"5min":9.9499999999999993,"15min":9.4789999999999992},"min":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"max":{"1min":27.917000000000002,"5min":27.917000000000002,"15min":27.917000000000002},"last":4.8390000000000004},{"interface":"front","average":{"1min":10.874000000000001,"5min":10.355,"15min":10.581},"min":{"1min":6.2930000000000001,"5min":5.1210000000000004,"15min":5.1210000000000004},"max":{"1min":30.068000000000001,"5min":30.068000000000001,"15min":30.068000000000001},"last":8.3070000000000004}]},{"osd":1,"last update":"Mon Oct 30 06:31:25 2023","interfaces":[{"interface":"back","average":{"1min":14.945,"5min":12.913,"15min":12.016},"min":{"1min":6.2359999999999998,"5min":6.2359999999999998,"15min":6.2359999999999998},"max":{"1min":61.877000000000002,"5min":61.877000000000002,"15min":61.877000000000002},"last":11.359999999999999},{"interface":"front","average":{"1min":13.856999999999999,"5min":13.173,"15min":13.930999999999999},"min":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"max":{"1min":62.469000000000001,"5min":62.469000000000001,"15min":62.469000000000001},"last":9.0449999999999999}]},{"osd":2,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.037000000000001,"5min":14.952999999999999,"15min":13.112},"min":{"1min":9.8680000000000003,"5min":7.5339999999999998,"15min":6.6200000000000001},"max":{"1min":24.238,"5min":26.809000000000001,"15min":26.809000000000001},"last":8.0129999999999999},{"interface":"front","average":{"1min":13.535,"5min":14.238,"15min":15.266999999999999},"min":{"1min":7.8470000000000004,"5min":7.8470000000000004,"15min":7.8470000000000004},"max":{"1min":22.445,"5min":34.122999999999998,"15min":34.122999999999998},"last":12.305999999999999}]},{"osd":3,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":9.7650000000000006,"5min":10.805999999999999,"15min":11.029999999999999},"min":{"1min":6.5570000000000004,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":29.73,"5min":66.722999999999999,"15min":66.722999999999999},"last":15.233000000000001},{"interface":"front","average":{"1min":16.341999999999999,"5min":16.515999999999998,"15min":16.108000000000001},"min":{"1min":8.3140000000000001,"5min":8.0389999999999997,"15min":8.0389999999999997},"max":{"1min":31.306000000000001,"5min":69.346000000000004,"15min":69.346000000000004},"last":12.91}]},{"osd":5,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":14.895,"5min":16.187000000000001,"15min":18.460000000000001},"min":{"1min":10.494,"5min":7.9560000000000004,"15min":6.4950000000000001},"max":{"1min":19.137,"5min":50.207000000000001,"15min":64.474999999999994},"last":10.82},{"interface":"front","average":{"1min":10.176,"5min":12.964,"15min":15.085000000000001},"min":{"1min":6.194,"5min":6.194,"15min":5.7770000000000001},"max":{"1min":15.859,"5min":48.356000000000002,"15min":75.980000000000004},"last":16.388999999999999}]},{"osd":6,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.033000000000001,"5min":17.939,"15min":18.611999999999998},"min":{"1min":10.064,"5min":9.0809999999999995,"15min":9.0809999999999995},"max":{"1min":23.018999999999998,"5min":60.633000000000003,"15min":60.633000000000003},"last":13.59},{"interface":"front","average":{"1min":12.702,"5min":13.997999999999999,"15min":14.025},"min":{"1min":8.5500000000000007,"5min":7.343,"15min":7.343},"max":{"1min":24.661000000000001,"5min":48.267000000000003,"15min":48.267000000000003},"last":17.492999999999999}]},{"osd":7,"last update":"Mon Oct 30 06:31:22 2023","interfaces":[{"interface":"back","average":{"1min":16.18,"5min":16.689,"15min":17.701000000000001},"min":{"1min":11.278,"5min":8.7590000000000003,"15min":3.5579999999999998},"max":{"1min":29.370000000000001,"5min":29.370000000000001,"15min":74.977999999999994},"last":18.93},{"interface":"front","average":{"1min":17.969999999999999,"5min":18.548999999999999,"15min":20.998999999999999},"min":{"1min":11.813000000000001,"5min":9.9429999999999996,"15min":5.0060000000000002},"max":{"1min":26.393000000000001,"5min":39.973999999999997,"15min":83.138000000000005},"last":20.178000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":397840,"data_compressed":689,"data_compressed_allocated":24576,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-30T06:32:20.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-30T06:32:21.161 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-30T06:32:21.161 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-30T06:32:21.162 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-30T06:32:21.162 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:32:21.172 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:32:21.163+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 86334) UID: 0 2023-10-30T06:32:21.172 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-10-30T06:32:21.164+0000 ac7d640 -1 osd.0 147 *** Got signal Terminated *** 2023-10-30T06:32:25.225 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:57.106 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.232 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:57.112 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.250 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:03:57.131 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.277 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:39.049 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.279 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:51.614 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.283 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:39.055 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.283 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:51.619 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.299 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:51.634 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:25.300 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:39.072 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:27.264 INFO:tasks.ceph.osd.0:Stopped 2023-10-30T06:32:27.265 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-30T06:32:27.265 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:32:27.274 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:32:27.264+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 86341) UID: 0 2023-10-30T06:32:27.274 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-10-30T06:32:27.265+0000 ac7d640 -1 osd.1 149 *** Got signal Terminated *** 2023-10-30T06:32:28.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-30T06:32:30.251 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:02.131 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.256 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:02.137 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.275 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:02.156 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.302 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:44.074 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.303 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:56.637 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.303 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:56.638 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.307 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:44.080 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.314 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:02:56.649 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:30.319 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:01:44.092 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:34.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-10-30T06:32:45.569 INFO:tasks.ceph.osd.1:Stopped 2023-10-30T06:32:45.570 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-30T06:32:45.570 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:32:45.578 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:32:45.570+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 86351) UID: 0 2023-10-30T06:32:45.578 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-10-30T06:32:45.570+0000 ac7d640 -1 osd.2 151 *** Got signal Terminated *** 2023-10-30T06:32:50.292 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:22.173 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.296 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:22.177 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.313 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:22.193 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.341 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.114 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.345 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:16.680 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.345 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.118 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.346 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:16.681 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.359 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:16.694 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:50.363 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:04.136 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:32:57.775 INFO:tasks.ceph.osd.2:Stopped 2023-10-30T06:32:57.775 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-30T06:32:57.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:32:57.786 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:32:57.776+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 86356) UID: 0 2023-10-30T06:32:57.787 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-10-30T06:32:57.776+0000 ac7d640 -1 osd.3 153 *** Got signal Terminated *** 2023-10-30T06:33:10.347 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:42.227 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.351 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:42.232 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.375 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:42.255 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.401 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:36.735 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.404 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:36.739 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.405 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:24.177 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.408 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:24.180 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.419 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:36.754 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:10.423 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:24.195 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.456 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:52.336 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.461 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:52.342 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.469 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:46.804 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.470 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:34.242 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.473 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:46.808 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.474 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:34.246 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.478 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:04:52.358 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.490 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:34.262 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:20.491 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:46.826 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.391 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:02.271 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.392 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:02.273 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.402 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:02.283 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.438 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:56.773 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.440 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:56.776 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.455 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:03:56.790 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.457 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:44.230 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.463 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:44.235 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:30.477 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:02:44.247 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:46.588 INFO:tasks.ceph.osd.3:Stopped 2023-10-30T06:33:46.589 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-30T06:33:46.589 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:33:46.598 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:33:46.589+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 86598) UID: 0 2023-10-30T06:33:46.599 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-10-30T06:33:46.589+0000 ac7d640 -1 osd.4 155 *** Got signal Terminated *** 2023-10-30T06:33:46.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-30T06:33:50.444 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:22.325 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.448 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:22.329 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.464 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:22.344 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.495 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:04.268 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.497 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:16.832 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.498 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:04.271 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.500 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:16.835 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.514 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:04.286 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:50.514 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:16.849 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:33:52.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-10-30T06:33:52.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-30T06:33:52.692 INFO:tasks.ceph.osd.4:Stopped 2023-10-30T06:33:52.692 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-30T06:33:52.693 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:33:52.704 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:33:52.692+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 86601) UID: 0 2023-10-30T06:33:52.705 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-10-30T06:33:52.693+0000 ac7d640 -1 osd.5 156 *** Got signal Terminated *** 2023-10-30T06:33:58.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-10-30T06:34:10.497 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:42.378 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.512 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:42.393 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.532 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:05:42.413 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.547 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:36.882 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.551 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:36.886 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.563 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:24.336 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.565 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:36.900 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.566 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:24.338 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:10.582 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:24.355 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:15.756 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-10-30T06:34:17.098 INFO:tasks.ceph.osd.5:Stopped 2023-10-30T06:34:17.099 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-30T06:34:17.099 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:34:17.108 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:34:17.099+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 86604) UID: 0 2023-10-30T06:34:17.109 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-10-30T06:34:17.099+0000 ac7d640 -1 osd.6 158 *** Got signal Terminated *** 2023-10-30T06:34:21.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-10-30T06:34:30.539 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:02.420 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.561 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:02.442 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.579 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:02.460 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.588 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:56.923 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.591 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:56.926 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.592 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:44.365 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.596 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:44.369 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.604 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:04:56.939 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:30.614 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:03:44.387 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.586 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:22.466 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.588 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:22.468 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.603 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:22.484 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.632 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:04.404 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.633 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:16.968 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.636 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:16.971 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.637 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:04.410 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.649 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:16.984 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:34:50.652 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:04.424 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:01.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-10-30T06:35:05.911 INFO:tasks.ceph.osd.6:Stopped 2023-10-30T06:35:05.912 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-30T06:35:05.913 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:05.922 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:35:05.912+0000 ac7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 86606) UID: 0 2023-10-30T06:35:05.924 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-10-30T06:35:05.912+0000 ac7d640 -1 osd.7 158 *** Got signal Terminated *** 2023-10-30T06:35:06.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-10-30T06:35:10.592 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:42.473 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.639 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:42.519 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.643 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:36.978 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.646 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:36.982 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.655 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:42.535 111973== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.663 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:36.998 106669== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.686 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:24.459 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.689 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:24.462 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:10.703 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:24.476 127164== Warning: unimplemented fcntl command: 1036 2023-10-30T06:35:12.015 INFO:tasks.ceph.osd.7:Stopped 2023-10-30T06:35:12.016 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-30T06:35:12.016 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-30T06:35:12.016 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:12.017 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:35:12.016+0000 7fd570c63640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 86029) UID: 0 2023-10-30T06:35:12.017 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-10-30T06:35:12.016+0000 7fd570c63640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-30T06:35:12.043 INFO:tasks.ceph.mgr.y:Stopped 2023-10-30T06:35:12.043 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-30T06:35:12.044 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:12.068 INFO:tasks.ceph.mgr.x:Stopped 2023-10-30T06:35:12.069 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-30T06:35:12.069 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-30T06:35:12.069 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:12.099 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:35:12.089+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 111963) UID: 0 2023-10-30T06:35:12.100 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-10-30T06:35:12.091+0000 9f64640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-30T06:35:12.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-10-30T06:35:12.573 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:44.454 111973== 2023-10-30T06:35:12.574 INFO:tasks.ceph.mon.a.smithi079.stderr:==00:00:06:44.454 111973== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:35:12.585 INFO:tasks.ceph.mon.a.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:35:17.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-30T06:35:17.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-10-30T06:35:18.171 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:35:18.172 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-10-30T06:35:18.172 INFO:tasks.ceph.mon.a:Stopped 2023-10-30T06:35:18.173 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-30T06:35:18.173 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:18.189 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:35:18.176+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 127127) UID: 0 2023-10-30T06:35:18.190 INFO:tasks.ceph.mon.c.smithi079.stderr:2023-10-30T06:35:18.177+0000 9f64640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-10-30T06:35:18.700 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:32.472 127164== 2023-10-30T06:35:18.700 INFO:tasks.ceph.mon.c.smithi079.stderr:==00:00:04:32.472 127164== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:35:18.712 INFO:tasks.ceph.mon.c.smithi079.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:35:22.515 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-30T06:35:22.516 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-30T06:35:24.275 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:35:24.276 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi079 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-10-30T06:35:24.276 INFO:tasks.ceph.mon.c:Stopped 2023-10-30T06:35:24.277 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-30T06:35:24.277 INFO:teuthology.orchestra.run:waiting for 300 2023-10-30T06:35:24.292 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:35:24.280+0000 9f64640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 106667) UID: 0 2023-10-30T06:35:24.292 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-10-30T06:35:24.281+0000 9f64640 -1 mon.b@1(probing) e1 *** Got Signal Terminated *** 2023-10-30T06:35:24.798 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:51.133 106669== 2023-10-30T06:35:24.799 INFO:tasks.ceph.mon.b.smithi112.stderr:==00:00:05:51.133 106669== Exit program on first error (--exit-on-first-error=yes) 2023-10-30T06:35:24.808 INFO:tasks.ceph.mon.b.smithi112.stderr:daemon-helper: command failed with exit status 42 2023-10-30T06:35:27.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-30T06:35:27.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-30T06:35:30.379 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-10-30T06:35:30.380 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-10-30T06:35:30.381 INFO:tasks.ceph.mon.b:Stopped 2023-10-30T06:35:30.381 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-30T06:35:30.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW OPS\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-10-30T06:35:30.421 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:35:30.421 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-30T06:35:30.543 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:35:30.543 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-30T06:35:30.600 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:35:30.600 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-30T06:35:30.658 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:35:30.659 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-30T06:35:30.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:35:30.719 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-30T06:35:30.845 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:35:30.845 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-30T06:35:30.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:35:30.902 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-30T06:35:30.976 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:35:30.977 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-30T06:35:31.033 INFO:tasks.ceph:Archiving mon data... 2023-10-30T06:35:31.036 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/data/mon.a.tgz 2023-10-30T06:35:31.036 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2023-10-30T06:35:31.052 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.cioVN9fi0X 2023-10-30T06:35:31.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cioVN9fi0X 2023-10-30T06:35:31.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.cioVN9fi0X 2023-10-30T06:35:31.283 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.cioVN9fi0X is 349KB 2023-10-30T06:35:31.349 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.cioVN9fi0X 2023-10-30T06:35:31.363 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/data/mon.c.tgz 2023-10-30T06:35:31.364 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2023-10-30T06:35:31.417 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.NcwS2Ft5JW 2023-10-30T06:35:31.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NcwS2Ft5JW 2023-10-30T06:35:31.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.NcwS2Ft5JW 2023-10-30T06:35:31.628 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.NcwS2Ft5JW is 349KB 2023-10-30T06:35:31.723 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.NcwS2Ft5JW 2023-10-30T06:35:31.737 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/data/mon.b.tgz 2023-10-30T06:35:31.738 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-10-30T06:35:31.757 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.bczsts8IO1 2023-10-30T06:35:31.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.bczsts8IO1 2023-10-30T06:35:31.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.bczsts8IO1 2023-10-30T06:35:32.011 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.bczsts8IO1 is 349KB 2023-10-30T06:35:32.115 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.bczsts8IO1 2023-10-30T06:35:32.130 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-30T06:35:32.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T06:35:32.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-30T06:35:32.197 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-30T06:35:32.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T06:35:32.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-30T06:35:32.240 INFO:teuthology.orchestra.run.smithi079.stdout:/var/log/ceph/valgrind/mon.a.log: Leak_StillReachable 2023-10-30T06:35:32.240 INFO:teuthology.orchestra.run.smithi079.stdout:/var/log/ceph/valgrind/mon.c.log: Leak_StillReachable 2023-10-30T06:35:32.241 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.a.log kind Leak_StillReachable 2023-10-30T06:35:32.242 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.a.log 2023-10-30T06:35:32.242 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.c.log kind Leak_StillReachable 2023-10-30T06:35:32.242 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.c.log 2023-10-30T06:35:32.278 INFO:teuthology.orchestra.run.smithi112.stdout:/var/log/ceph/valgrind/mon.b.log: Leak_StillReachable 2023-10-30T06:35:32.279 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mon.b.log kind Leak_StillReachable 2023-10-30T06:35:32.279 ERROR:tasks.ceph:saw valgrind issue Leak_StillReachable in /var/log/ceph/valgrind/mon.b.log 2023-10-30T06:35:32.279 INFO:tasks.ceph:Archiving crash dumps... 2023-10-30T06:35:32.282 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi079/crash 2023-10-30T06:35:32.283 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T06:35:32.310 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi112/crash 2023-10-30T06:35:32.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-30T06:35:32.345 INFO:tasks.ceph:Compressing logs... 2023-10-30T06:35:32.345 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T06:35:32.354 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-30T06:35:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-30T06:35:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86378.log 2023-10-30T06:35:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86509.log 2023-10-30T06:35:32.429 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86378.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2023-10-30T06:35:32.429 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86378.log.gz 2023-10-30T06:35:32.429 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86509.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2023-10-30T06:35:32.429 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86509.log.gz 2023-10-30T06:35:32.429 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/valgrind/osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2023-10-30T06:35:32.430 INFO:teuthology.orchestra.run.smithi112.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2023-10-30T06:35:32.430 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2023-10-30T06:35:32.430 INFO:teuthology.orchestra.run.smithi112.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2023-10-30T06:35:32.430 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/valgrind/osd.6.log: -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2023-10-30T06:35:32.430 INFO:teuthology.orchestra.run.smithi112.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2023-10-30T06:35:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T06:35:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/valgrind/osd.4.log: 74.7% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2023-10-30T06:35:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-30T06:35:32.431 INFO:teuthology.orchestra.run.smithi112.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2023-10-30T06:35:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86418.log 2023-10-30T06:35:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86338.log 2023-10-30T06:35:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86218.log 2023-10-30T06:35:32.432 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86418.log.gz 2023-10-30T06:35:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86258.log 2023-10-30T06:35:32.432 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86338.log.gz 2023-10-30T06:35:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-30T06:35:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86218.log.gz 2023-10-30T06:35:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86258.log.gz 2023-10-30T06:35:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-30T06:35:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86469.log 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86298.log 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86469.log.gz 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T06:35:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.86298.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86298.log.gz 2023-10-30T06:35:32.435 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-30T06:35:32.435 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: 88.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-30T06:35:32.437 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-30T06:35:32.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-30T06:35:32.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120695.log 2023-10-30T06:35:32.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109245.log 2023-10-30T06:35:32.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108929.log 2023-10-30T06:35:32.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120695.log: /var/log/ceph/ceph-client.admin.109245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120695.log.gz 2023-10-30T06:35:32.547 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109145.log 2023-10-30T06:35:32.547 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109245.log.gz 2023-10-30T06:35:32.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132568.log 2023-10-30T06:35:32.547 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108929.log.gz 2023-10-30T06:35:32.548 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.109145.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.91136.log 2023-10-30T06:35:32.548 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109145.log.gz 2023-10-30T06:35:32.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115494.log 2023-10-30T06:35:32.548 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132568.log.gz 2023-10-30T06:35:32.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91136.log.gz 2023-10-30T06:35:32.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131145.log 2023-10-30T06:35:32.549 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118665.log 2023-10-30T06:35:32.549 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115494.log.gz 2023-10-30T06:35:32.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.131145.log: /var/log/ceph/ceph-mon.a.log 2023-10-30T06:35:32.549 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131145.log.gz 2023-10-30T06:35:32.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108605.log 2023-10-30T06:35:32.550 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118665.log.gz 2023-10-30T06:35:32.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108045.log 2023-10-30T06:35:32.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.108605.log: -- /var/log/ceph/ceph-client.admin.121771.log 2023-10-30T06:35:32.550 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108605.log.gz 2023-10-30T06:35:32.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96945.log 2023-10-30T06:35:32.551 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108045.log.gz 2023-10-30T06:35:32.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105367.log 2023-10-30T06:35:32.551 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121771.log.gz 2023-10-30T06:35:32.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113315.log 2023-10-30T06:35:32.552 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96945.log.gz 2023-10-30T06:35:32.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92928.log 2023-10-30T06:35:32.552 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105367.log.gz 2023-10-30T06:35:32.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129460.log 2023-10-30T06:35:32.552 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113315.log.gz 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102248.log 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92928.log.gz 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104717.log 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129460.log.gz 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91872.log 2023-10-30T06:35:32.553 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102248.log.gz 2023-10-30T06:35:32.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117220.log 2023-10-30T06:35:32.554 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104717.log.gz 2023-10-30T06:35:32.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117429.log 2023-10-30T06:35:32.554 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91872.log.gz 2023-10-30T06:35:32.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116819.log 2023-10-30T06:35:32.555 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117220.log.gz 2023-10-30T06:35:32.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127737.log 2023-10-30T06:35:32.555 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117429.log.gz 2023-10-30T06:35:32.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132610.log 2023-10-30T06:35:32.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116819.log.gz 2023-10-30T06:35:32.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132736.log 2023-10-30T06:35:32.556 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127737.log.gz 2023-10-30T06:35:32.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124091.log 2023-10-30T06:35:32.556 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132610.log.gz 2023-10-30T06:35:32.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126871.log 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132736.log.gz 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120957.log 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124091.log.gz 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130774.log 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126871.log.gz 2023-10-30T06:35:32.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117588.log 2023-10-30T06:35:32.558 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120957.log.gz 2023-10-30T06:35:32.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90678.log 2023-10-30T06:35:32.558 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130774.log.gz 2023-10-30T06:35:32.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127574.log 2023-10-30T06:35:32.558 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117588.log.gz 2023-10-30T06:35:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130048.log 2023-10-30T06:35:32.559 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90678.log.gz 2023-10-30T06:35:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119533.log 2023-10-30T06:35:32.559 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127574.log.gz 2023-10-30T06:35:32.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105347.log 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130048.log.gz 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109077.log 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119533.log.gz 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105347.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.131812.log -- replaced with /var/log/ceph/ceph-client.admin.105347.log.gz 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130090.log 2023-10-30T06:35:32.561 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109077.log.gz 2023-10-30T06:35:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92188.log 2023-10-30T06:35:32.561 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131812.log.gz 2023-10-30T06:35:32.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118541.log 2023-10-30T06:35:32.561 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130090.log.gz 2023-10-30T06:35:32.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119140.log 2023-10-30T06:35:32.562 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92188.log.gz 2023-10-30T06:35:32.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90922.log 2023-10-30T06:35:32.562 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118541.log.gz 2023-10-30T06:35:32.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120014.log 2023-10-30T06:35:32.563 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119140.log.gz 2023-10-30T06:35:32.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105730.log 2023-10-30T06:35:32.563 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90922.log.gz 2023-10-30T06:35:32.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95207.log 2023-10-30T06:35:32.563 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120014.log.gz 2023-10-30T06:35:32.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115222.log 2023-10-30T06:35:32.564 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105730.log.gz 2023-10-30T06:35:32.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120488.log 2023-10-30T06:35:32.564 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95207.log.gz 2023-10-30T06:35:32.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130776.log 2023-10-30T06:35:32.565 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115222.log.gz 2023-10-30T06:35:32.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90380.log 2023-10-30T06:35:32.565 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120488.log.gz 2023-10-30T06:35:32.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94941.log 2023-10-30T06:35:32.565 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130776.log.gz 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130006.log 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90380.log.gz 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125531.log 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94941.log.gz 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127131.log 2023-10-30T06:35:32.566 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130006.log.gz 2023-10-30T06:35:32.567 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99280.log 2023-10-30T06:35:32.567 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125531.log.gz 2023-10-30T06:35:32.567 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130964.log 2023-10-30T06:35:32.567 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127131.log.gz 2023-10-30T06:35:32.567 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97066.log 2023-10-30T06:35:32.568 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99280.log.gz 2023-10-30T06:35:32.568 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117125.log 2023-10-30T06:35:32.568 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130964.log.gz 2023-10-30T06:35:32.568 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123328.log 2023-10-30T06:35:32.568 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97066.log.gz 2023-10-30T06:35:32.569 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102805.log 2023-10-30T06:35:32.569 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117125.log.gz 2023-10-30T06:35:32.569 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131072.log 2023-10-30T06:35:32.569 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123328.log.gz 2023-10-30T06:35:32.569 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102805.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2023-10-30T06:35:32.570 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102805.log.gz 2023-10-30T06:35:32.570 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131072.log: gzip -5 --verbose -- 0.0% /var/log/ceph/valgrind/osd.2.log 2023-10-30T06:35:32.570 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.131072.log.gz 2023-10-30T06:35:32.570 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2023-10-30T06:35:32.570 INFO:teuthology.orchestra.run.smithi079.stderr: 74.8% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr: 63.4% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2023-10-30T06:35:32.571 INFO:teuthology.orchestra.run.smithi079.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2023-10-30T06:35:32.572 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115598.log 2023-10-30T06:35:32.572 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/valgrind/osd.3.log: 74.7% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2023-10-30T06:35:32.572 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115993.log 2023-10-30T06:35:32.572 INFO:teuthology.orchestra.run.smithi079.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2023-10-30T06:35:32.572 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125636.log 2023-10-30T06:35:32.573 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115598.log.gz 2023-10-30T06:35:32.573 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132148.log 2023-10-30T06:35:32.573 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115993.log.gz 2023-10-30T06:35:32.573 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125636.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.125636.log.gz -5 2023-10-30T06:35:32.573 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126058.log 2023-10-30T06:35:32.574 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95975.log 2023-10-30T06:35:32.574 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132148.log.gz 2023-10-30T06:35:32.574 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100703.log 2023-10-30T06:35:32.574 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126058.log.gz 2023-10-30T06:35:32.574 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95975.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.95975.log.gz -- 2023-10-30T06:35:32.575 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.90112.log 2023-10-30T06:35:32.575 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103727.log 2023-10-30T06:35:32.575 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100703.log.gz 2023-10-30T06:35:32.575 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129922.log 2023-10-30T06:35:32.575 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90112.log.gz 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119392.log 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103727.log.gz 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100134.log 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129922.log.gz 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89956.log/var/log/ceph/ceph-client.admin.119392.log: 2023-10-30T06:35:32.576 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119392.log.gz 2023-10-30T06:35:32.577 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100134.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.100134.log.gz -5 2023-10-30T06:35:32.577 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96699.log 2023-10-30T06:35:32.577 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.89956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130768.log 2023-10-30T06:35:32.577 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89956.log.gz 2023-10-30T06:35:32.577 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130767.log 2023-10-30T06:35:32.578 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96699.log.gz 2023-10-30T06:35:32.578 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131704.log 2023-10-30T06:35:32.578 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130768.log.gz 2023-10-30T06:35:32.578 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114740.log 2023-10-30T06:35:32.578 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130767.log.gz 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132400.log 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131704.log.gz 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97386.log 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114740.log.gz 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125448.log 2023-10-30T06:35:32.579 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132400.log.gz 2023-10-30T06:35:32.580 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90382.log 2023-10-30T06:35:32.580 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97386.log.gz 2023-10-30T06:35:32.580 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132652.log 2023-10-30T06:35:32.580 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125448.log.gz 2023-10-30T06:35:32.580 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117913.log 2023-10-30T06:35:32.581 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90382.log.gz 2023-10-30T06:35:32.581 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130552.log 2023-10-30T06:35:32.581 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132652.log.gz 2023-10-30T06:35:32.581 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117987.log 0.0% 2023-10-30T06:35:32.581 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117913.log.gz 2023-10-30T06:35:32.582 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132778.log 2023-10-30T06:35:32.582 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130552.log.gz 2023-10-30T06:35:32.582 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89891.log 2023-10-30T06:35:32.582 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117987.log.gz 2023-10-30T06:35:32.582 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95590.log 2023-10-30T06:35:32.583 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132778.log.gz 2023-10-30T06:35:32.583 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.89891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113090.log 2023-10-30T06:35:32.583 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89891.log.gz 2023-10-30T06:35:32.583 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126446.log 2023-10-30T06:35:32.583 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95590.log.gz 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113090.log.gz 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126614.log 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129575.log 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126446.log.gz 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113472.log 2023-10-30T06:35:32.584 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126614.log.gz 2023-10-30T06:35:32.585 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115783.log 2023-10-30T06:35:32.585 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129575.log.gz 2023-10-30T06:35:32.585 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93270.log 2023-10-30T06:35:32.585 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113472.log.gz 2023-10-30T06:35:32.585 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130300.log 2023-10-30T06:35:32.586 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115783.log.gz 2023-10-30T06:35:32.586 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.93270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131644.log 2023-10-30T06:35:32.586 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93270.log.gz 2023-10-30T06:35:32.586 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100723.log 2023-10-30T06:35:32.586 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130300.log.gz 2023-10-30T06:35:32.587 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131764.log 2023-10-30T06:35:32.587 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131644.log.gz 2023-10-30T06:35:32.587 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.100723.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.86361.log 2023-10-30T06:35:32.587 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100723.log.gz 2023-10-30T06:35:32.587 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109369.log 2023-10-30T06:35:32.588 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131764.log.gz 2023-10-30T06:35:32.588 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.86361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103083.log 2023-10-30T06:35:32.588 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86361.log.gz 2023-10-30T06:35:32.588 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132106.log 2023-10-30T06:35:32.588 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109369.log.gz 2023-10-30T06:35:32.589 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124332.log 2023-10-30T06:35:32.589 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103083.log.gz 2023-10-30T06:35:32.589 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91810.log 2023-10-30T06:35:32.589 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132106.log.gz 2023-10-30T06:35:32.589 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113120.log 2023-10-30T06:35:32.590 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124332.log.gz 2023-10-30T06:35:32.590 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105205.log 2023-10-30T06:35:32.590 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91810.log.gz 2023-10-30T06:35:32.590 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90384.log 2023-10-30T06:35:32.590 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113120.log.gz 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124172.log 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105205.log.gz 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90384.log.gz 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102321.log 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115528.log 2023-10-30T06:35:32.591 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124172.log.gz 2023-10-30T06:35:32.592 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98868.log 2023-10-30T06:35:32.592 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102321.log.gz 2023-10-30T06:35:32.592 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100260.log 2023-10-30T06:35:32.592 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115528.log.gz 2023-10-30T06:35:32.592 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98868.log.gz 2023-10-30T06:35:32.593 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128184.log 2023-10-30T06:35:32.593 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112702.log 2023-10-30T06:35:32.593 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100260.log.gz 2023-10-30T06:35:32.593 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128184.log.gz 2023-10-30T06:35:32.593 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117708.log 2023-10-30T06:35:32.594 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112702.log: /var/log/ceph/ceph-client.admin.90063.log 2023-10-30T06:35:32.594 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112702.log.gz 2023-10-30T06:35:32.594 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117708.log.gz 2023-10-30T06:35:32.594 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126978.log 2023-10-30T06:35:32.594 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91640.log 2023-10-30T06:35:32.595 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90063.log.gz 2023-10-30T06:35:32.595 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126727.log 2023-10-30T06:35:32.595 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126978.log.gz 2023-10-30T06:35:32.595 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106786.log 2023-10-30T06:35:32.595 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91640.log.gz 2023-10-30T06:35:32.596 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126727.log.gz 2023-10-30T06:35:32.596 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96441.log 2023-10-30T06:35:32.596 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108652.log 2023-10-30T06:35:32.596 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.106786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106786.log.gz 2023-10-30T06:35:32.596 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99069.log 2023-10-30T06:35:32.597 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96441.log.gz 2023-10-30T06:35:32.597 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95935.log 2023-10-30T06:35:32.597 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108652.log.gz 2023-10-30T06:35:32.597 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91444.log 2023-10-30T06:35:32.597 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99069.log.gz 2023-10-30T06:35:32.598 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114972.log 2023-10-30T06:35:32.598 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95935.log.gz 2023-10-30T06:35:32.598 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100653.log 2023-10-30T06:35:32.598 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91444.log.gz 2023-10-30T06:35:32.598 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94856.log 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114972.log.gz 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129440.log 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.100653.log.gz 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128354.log 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94856.log.gz 2023-10-30T06:35:32.599 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115190.log 2023-10-30T06:35:32.600 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129440.log.gz 2023-10-30T06:35:32.600 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122422.log 2023-10-30T06:35:32.600 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128354.log.gz 2023-10-30T06:35:32.600 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99792.log 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115190.log.gz 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.122422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115619.log 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122422.log.gz 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131408.log 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99792.log.gz 2023-10-30T06:35:32.601 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98671.log 2023-10-30T06:35:32.602 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115619.log.gz 2023-10-30T06:35:32.602 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126771.log 2023-10-30T06:35:32.602 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131408.log.gz 2023-10-30T06:35:32.602 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90204.log 2023-10-30T06:35:32.602 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98671.log.gz 2023-10-30T06:35:32.603 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129833.log 2023-10-30T06:35:32.603 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126771.log.gz 2023-10-30T06:35:32.603 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131938.log 2023-10-30T06:35:32.603 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90204.log.gz 2023-10-30T06:35:32.603 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103125.log 2023-10-30T06:35:32.604 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129833.log.gz 2023-10-30T06:35:32.604 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115394.log 2023-10-30T06:35:32.604 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131938.log.gz 2023-10-30T06:35:32.604 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127843.log 2023-10-30T06:35:32.604 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103125.log.gz 2023-10-30T06:35:32.605 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125286.log 2023-10-30T06:35:32.605 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115394.log.gz 2023-10-30T06:35:32.605 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92012.log 2023-10-30T06:35:32.605 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127843.log.gz 2023-10-30T06:35:32.605 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129665.log 2023-10-30T06:35:32.606 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125286.log.gz 2023-10-30T06:35:32.606 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131519.log 2023-10-30T06:35:32.606 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92012.log.gz 2023-10-30T06:35:32.606 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129791.log 2023-10-30T06:35:32.606 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129665.log.gz 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127822.log 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131519.log.gz 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131577.log 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129791.log.gz 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115863.log 2023-10-30T06:35:32.607 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127822.log.gz 2023-10-30T06:35:32.608 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112853.log 2023-10-30T06:35:32.608 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131577.log.gz 2023-10-30T06:35:32.608 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123004.log 2023-10-30T06:35:32.608 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115863.log.gz 2023-10-30T06:35:32.608 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132022.log 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112853.log.gz 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104903.log 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123004.log.gz 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113512.log 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132022.log.gz 2023-10-30T06:35:32.609 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118175.log 2023-10-30T06:35:32.610 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104903.log.gz 2023-10-30T06:35:32.610 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132274.log 2023-10-30T06:35:32.610 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113512.log.gz 2023-10-30T06:35:32.610 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104152.log 2023-10-30T06:35:32.610 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118175.log.gz 2023-10-30T06:35:32.611 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105307.log 2023-10-30T06:35:32.611 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132274.log.gz 2023-10-30T06:35:32.611 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123054.log 2023-10-30T06:35:32.611 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104152.log.gz 2023-10-30T06:35:32.611 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95086.log 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105307.log.gz 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117893.log 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123054.log.gz 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95086.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95086.log.gz 2023-10-30T06:35:32.612 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93019.log 2023-10-30T06:35:32.613 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117893.log.gz 2023-10-30T06:35:32.613 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95496.log 2023-10-30T06:35:32.613 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.93019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123228.log 2023-10-30T06:35:32.613 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93019.log.gz 2023-10-30T06:35:32.613 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116188.log 2023-10-30T06:35:32.614 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95496.log.gz 2023-10-30T06:35:32.614 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117345.log 2023-10-30T06:35:32.614 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123228.log.gz 2023-10-30T06:35:32.614 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116188.log.gz 2023-10-30T06:35:32.614 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90598.log 2023-10-30T06:35:32.615 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121898.log 2023-10-30T06:35:32.615 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117345.log.gz 2023-10-30T06:35:32.615 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123459.log 2023-10-30T06:35:32.615 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90598.log.gz 2023-10-30T06:35:32.615 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93692.log 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121898.log.gz 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94535.log 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123459.log.gz 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.93692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96141.log 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93692.log.gz 2023-10-30T06:35:32.616 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129623.log 2023-10-30T06:35:32.617 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94535.log.gz 2023-10-30T06:35:32.617 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113596.log 2023-10-30T06:35:32.617 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96141.log.gz 2023-10-30T06:35:32.617 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91767.log 2023-10-30T06:35:32.617 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129623.log.gz 2023-10-30T06:35:32.618 INFO:teuthology.orchestra.run.smithi079.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-30T06:35:32.618 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123462.log 2023-10-30T06:35:32.618 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113596.log.gz 2023-10-30T06:35:32.618 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112121.log 2023-10-30T06:35:32.618 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91767.log.gz 2023-10-30T06:35:32.619 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131854.log 2023-10-30T06:35:32.619 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123462.log.gz 2023-10-30T06:35:32.619 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132064.log 2023-10-30T06:35:32.619 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112121.log.gz 2023-10-30T06:35:32.619 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96781.log 2023-10-30T06:35:32.620 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131854.log.gz 2023-10-30T06:35:32.620 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103444.log 2023-10-30T06:35:32.620 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132064.log.gz 2023-10-30T06:35:32.620 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109567.log 2023-10-30T06:35:32.620 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96781.log.gz 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90669.log 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103444.log.gz 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132526.log 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109567.log.gz 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123663.log 2023-10-30T06:35:32.621 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90669.log.gz 2023-10-30T06:35:32.622 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108368.log 2023-10-30T06:35:32.622 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132526.log.gz 2023-10-30T06:35:32.622 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111967.log 2023-10-30T06:35:32.622 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123663.log.gz 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126142.log 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108368.log.gz 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.111967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104963.log 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.111967.log.gz 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130384.log 2023-10-30T06:35:32.623 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126142.log.gz 2023-10-30T06:35:32.624 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123746.log 2023-10-30T06:35:32.624 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104963.log.gz 2023-10-30T06:35:32.624 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91362.log 2023-10-30T06:35:32.624 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130384.log.gz 2023-10-30T06:35:32.624 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131478.log 2023-10-30T06:35:32.625 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123746.log.gz 2023-10-30T06:35:32.625 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121338.log 2023-10-30T06:35:32.625 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91362.log.gz 2023-10-30T06:35:32.625 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125615.log 2023-10-30T06:35:32.625 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131478.log.gz 2023-10-30T06:35:32.626 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91682.log 2023-10-30T06:35:32.626 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121338.log.gz 2023-10-30T06:35:32.626 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118327.log 2023-10-30T06:35:32.626 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125615.log.gz 2023-10-30T06:35:32.626 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128144.log 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91682.log.gz 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123075.log 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118327.log.gz 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102744.log 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128144.log.gz 2023-10-30T06:35:32.627 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105265.log 2023-10-30T06:35:32.628 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123075.log.gz 2023-10-30T06:35:32.628 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109670.log 2023-10-30T06:35:32.628 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102744.log.gz 2023-10-30T06:35:32.628 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122140.log 2023-10-30T06:35:32.628 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105265.log.gz 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127684.log 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109670.log.gz 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.122140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122823.log 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122140.log.gz 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132860.log 2023-10-30T06:35:32.629 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127684.log.gz 2023-10-30T06:35:32.630 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.122823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104196.log 2023-10-30T06:35:32.630 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122823.log.gz 2023-10-30T06:35:32.630 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121617.log 2023-10-30T06:35:32.630 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132860.log.gz 2023-10-30T06:35:32.630 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95423.log 2023-10-30T06:35:32.631 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104196.log.gz 2023-10-30T06:35:32.631 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105887.log 2023-10-30T06:35:32.631 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121617.log.gz 2023-10-30T06:35:32.631 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95423.log.gz 2023-10-30T06:35:32.631 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120054.log 2023-10-30T06:35:32.632 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120758.log 2023-10-30T06:35:32.632 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105887.log.gz 2023-10-30T06:35:32.632 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109937.log 2023-10-30T06:35:32.632 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120054.log.gz 2023-10-30T06:35:32.632 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103948.log 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120758.log.gz 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96015.log 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109937.log.gz 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116364.log 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103948.log.gz 2023-10-30T06:35:32.633 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90386.log 2023-10-30T06:35:32.634 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96015.log.gz 2023-10-30T06:35:32.634 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90377.log 2023-10-30T06:35:32.634 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116364.log.gz 2023-10-30T06:35:32.634 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91022.log 2023-10-30T06:35:32.634 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90386.log.gz 2023-10-30T06:35:32.635 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124267.log 2023-10-30T06:35:32.635 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90377.log.gz 2023-10-30T06:35:32.635 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119067.log 2023-10-30T06:35:32.635 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91022.log.gz 2023-10-30T06:35:32.635 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94391.log 2023-10-30T06:35:32.636 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124267.log.gz 2023-10-30T06:35:32.636 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96599.log 2023-10-30T06:35:32.636 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119067.log.gz 2023-10-30T06:35:32.636 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95041.log 2023-10-30T06:35:32.636 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94391.log.gz 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96599.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96599.log.gz 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92484.log 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95041.log.gz 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131308.log 2023-10-30T06:35:32.637 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117748.log 2023-10-30T06:35:32.638 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92484.log.gz 2023-10-30T06:35:32.638 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112544.log 2023-10-30T06:35:32.638 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131308.log.gz 2023-10-30T06:35:32.638 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109731.log 2023-10-30T06:35:32.638 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117748.log.gz 2023-10-30T06:35:32.639 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93089.log 2023-10-30T06:35:32.639 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112544.log.gz 2023-10-30T06:35:32.639 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109731.log.gz 2023-10-30T06:35:32.639 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92651.log 2023-10-30T06:35:32.639 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.93089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110787.log 2023-10-30T06:35:32.640 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93089.log.gz 2023-10-30T06:35:32.640 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100196.log 2023-10-30T06:35:32.640 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92651.log.gz 2023-10-30T06:35:32.640 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.110787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131684.log 2023-10-30T06:35:32.640 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110787.log.gz 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94575.log 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100196.log.gz 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108743.log 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131684.log.gz 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114741.log 2023-10-30T06:35:32.641 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94575.log.gz 2023-10-30T06:35:32.642 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112617.log 2023-10-30T06:35:32.642 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108743.log.gz 2023-10-30T06:35:32.642 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126526.log 2023-10-30T06:35:32.642 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114741.log.gz 2023-10-30T06:35:32.642 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112617.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112617.log.gz 2023-10-30T06:35:32.643 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.91342.log 2023-10-30T06:35:32.643 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114951.log 2023-10-30T06:35:32.643 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126526.log.gz 2023-10-30T06:35:32.643 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105074.log 2023-10-30T06:35:32.643 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91342.log.gz 2023-10-30T06:35:32.644 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114951.log.gz 2023-10-30T06:35:32.644 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100536.log 2023-10-30T06:35:32.644 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131063.log 2023-10-30T06:35:32.644 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105074.log.gz 2023-10-30T06:35:32.644 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104748.log 2023-10-30T06:35:32.645 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100536.log.gz 2023-10-30T06:35:32.645 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92608.log 2023-10-30T06:35:32.645 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131063.log.gz 2023-10-30T06:35:32.645 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104748.log.gz 2023-10-30T06:35:32.645 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131896.log 2023-10-30T06:35:32.646 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128575.log 2023-10-30T06:35:32.646 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92608.log.gz 2023-10-30T06:35:32.646 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124333.log 2023-10-30T06:35:32.646 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131896.log.gz 2023-10-30T06:35:32.646 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91724.log 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128575.log.gz 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126057.log 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124333.log.gz 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103667.log 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91724.log.gz 2023-10-30T06:35:32.647 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127092.log 2023-10-30T06:35:32.648 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126057.log.gz 2023-10-30T06:35:32.648 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130510.log 2023-10-30T06:35:32.648 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103667.log.gz 2023-10-30T06:35:32.648 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126210.log 2023-10-30T06:35:32.648 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127092.log.gz 2023-10-30T06:35:32.649 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98026.log 2023-10-30T06:35:32.649 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130510.log.gz 2023-10-30T06:35:32.649 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116084.log 2023-10-30T06:35:32.649 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126210.log.gz 2023-10-30T06:35:32.649 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86550.log 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98026.log.gz 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116084.log.gz 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130258.log 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.86550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98447.log 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86550.log.gz 2023-10-30T06:35:32.650 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112411.log 2023-10-30T06:35:32.651 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130258.log.gz 2023-10-30T06:35:32.651 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90702.log 2023-10-30T06:35:32.651 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98447.log.gz 2023-10-30T06:35:32.651 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98799.log 2023-10-30T06:35:32.651 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112411.log.gz 2023-10-30T06:35:32.652 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92567.log 2023-10-30T06:35:32.652 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90702.log.gz 2023-10-30T06:35:32.652 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98799.log.gz 2023-10-30T06:35:32.652 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131043.log 2023-10-30T06:35:32.652 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92010.log 2023-10-30T06:35:32.653 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92567.log.gz 2023-10-30T06:35:32.653 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116312.log 2023-10-30T06:35:32.653 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131043.log.gz 2023-10-30T06:35:32.653 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118896.log 2023-10-30T06:35:32.653 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92010.log.gz 2023-10-30T06:35:32.654 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104248.log 2023-10-30T06:35:32.654 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116312.log.gz 2023-10-30T06:35:32.654 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124517.log 2023-10-30T06:35:32.654 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118896.log.gz 2023-10-30T06:35:32.654 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91598.log 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104248.log.gz 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97237.log 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124517.log.gz 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131139.log 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91598.log.gz 2023-10-30T06:35:32.655 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92015.log 2023-10-30T06:35:32.656 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97237.log.gz 2023-10-30T06:35:32.656 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131139.log.gz 2023-10-30T06:35:32.656 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127256.log 2023-10-30T06:35:32.656 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128252.log 2023-10-30T06:35:32.656 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92015.log.gz 2023-10-30T06:35:32.657 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127256.log.gz 2023-10-30T06:35:32.657 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94674.log 2023-10-30T06:35:32.657 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129058.log 2023-10-30T06:35:32.657 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128252.log.gz 2023-10-30T06:35:32.657 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.94674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130342.log 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94674.log.gz 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125975.log 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129058.log.gz 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102249.log 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130342.log.gz 2023-10-30T06:35:32.658 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132190.log 2023-10-30T06:35:32.659 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125975.log.gz 2023-10-30T06:35:32.659 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102249.log.gz 2023-10-30T06:35:32.659 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114158.log 2023-10-30T06:35:32.659 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90675.log 2023-10-30T06:35:32.659 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132190.log.gz 2023-10-30T06:35:32.660 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114824.log 2023-10-30T06:35:32.660 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114158.log.gz 2023-10-30T06:35:32.660 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90675.log.gz 2023-10-30T06:35:32.660 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108169.log 2023-10-30T06:35:32.660 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104495.log 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114824.log.gz 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97683.log 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108169.log.gz 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124656.log 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104495.log.gz 2023-10-30T06:35:32.661 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97683.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.97683.log.gz 2023-10-30T06:35:32.662 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.113976.log 2023-10-30T06:35:32.662 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98195.log 2023-10-30T06:35:32.662 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124656.log.gz 2023-10-30T06:35:32.662 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113976.log.gz 2023-10-30T06:35:32.662 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115763.log 2023-10-30T06:35:32.663 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132232.log 2023-10-30T06:35:32.663 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98195.log.gz 2023-10-30T06:35:32.663 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125833.log 2023-10-30T06:35:32.663 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115763.log.gz 2023-10-30T06:35:32.663 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124456.log 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132232.log.gz 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118495.log 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125833.log.gz 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120395.log 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124456.log.gz 2023-10-30T06:35:32.664 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86388.log 2023-10-30T06:35:32.665 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118495.log.gz 2023-10-30T06:35:32.665 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120395.log.gz 2023-10-30T06:35:32.665 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-30T06:35:32.665 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.86388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112091.log 2023-10-30T06:35:32.665 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86388.log.gz 2023-10-30T06:35:32.666 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116251.log 2023-10-30T06:35:32.666 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112091.log.gz 2023-10-30T06:35:32.666 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116474.log 2023-10-30T06:35:32.666 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90960.log 2023-10-30T06:35:32.666 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116251.log.gz 2023-10-30T06:35:32.667 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116781.log 2023-10-30T06:35:32.667 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116474.log.gz 2023-10-30T06:35:32.667 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127474.log 0.0% 2023-10-30T06:35:32.667 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90960.log.gz 2023-10-30T06:35:32.667 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98982.log 2023-10-30T06:35:32.668 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116781.log.gz 2023-10-30T06:35:32.668 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87082.log 2023-10-30T06:35:32.668 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127474.log.gz 2023-10-30T06:35:32.668 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131062.log 2023-10-30T06:35:32.668 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98982.log.gz 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98610.log 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.87082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87082.log.gz 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88202.log 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131062.log.gz 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96861.log 2023-10-30T06:35:32.669 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98610.log.gz 2023-10-30T06:35:32.670 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91072.log 2023-10-30T06:35:32.670 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.88202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88202.log.gz 2023-10-30T06:35:32.670 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129880.log 2023-10-30T06:35:32.670 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96861.log.gz 2023-10-30T06:35:32.670 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104617.log 2023-10-30T06:35:32.671 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91072.log.gz 2023-10-30T06:35:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103364.log 2023-10-30T06:35:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129880.log.gz 2023-10-30T06:35:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104617.log.gz 2023-10-30T06:35:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128677.log 2023-10-30T06:35:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130426.log 2023-10-30T06:35:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103364.log.gz 2023-10-30T06:35:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120435.log 2023-10-30T06:35:32.672 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128677.log.gz 2023-10-30T06:35:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129964.log 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130426.log.gz 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5/var/log/ceph/ceph-client.admin.120435.log: --verbose -- /var/log/ceph/ceph-client.admin.92848.log 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120435.log.gz 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123706.log 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129964.log.gz 2023-10-30T06:35:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130174.log 2023-10-30T06:35:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92848.log.gz 2023-10-30T06:35:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125307.log 2023-10-30T06:35:32.674 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123706.log.gz 2023-10-30T06:35:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97743.log 2023-10-30T06:35:32.674 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130174.log.gz 2023-10-30T06:35:32.675 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.125307.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.97124.log 2023-10-30T06:35:32.675 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125307.log.gz 2023-10-30T06:35:32.675 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108217.log 2023-10-30T06:35:32.675 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97743.log.gz 2023-10-30T06:35:32.675 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.97124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116532.log 2023-10-30T06:35:32.676 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97124.log.gz 2023-10-30T06:35:32.676 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114906.log 2023-10-30T06:35:32.676 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108217.log.gz 2023-10-30T06:35:32.676 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131348.log 2023-10-30T06:35:32.676 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116532.log.gz 2023-10-30T06:35:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99414.log/var/log/ceph/ceph-client.admin.114906.log: 2023-10-30T06:35:32.677 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114906.log.gz 2023-10-30T06:35:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131348.log.gz 2023-10-30T06:35:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115462.log 2023-10-30T06:35:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120094.log 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99414.log.gz 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.115462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115462.log.gz 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130594.log 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.120094.log: /var/log/ceph/ceph-client.admin.119653.log 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120094.log.gz 2023-10-30T06:35:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130594.log.gz 2023-10-30T06:35:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121816.log 2023-10-30T06:35:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103603.log 2023-10-30T06:35:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119653.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119653.log.gz 2023-10-30T06:35:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89476.log 2023-10-30T06:35:32.679 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121816.log.gz 2023-10-30T06:35:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116922.log 2023-10-30T06:35:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103603.log.gz 2023-10-30T06:35:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.89476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100976.log 2023-10-30T06:35:32.680 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89476.log.gz 2023-10-30T06:35:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113552.log 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116922.log.gz 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96358.log 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100976.log.gz 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113552.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113552.log.gz 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114540.log 2023-10-30T06:35:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.96358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126687.log 2023-10-30T06:35:32.682 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96358.log.gz 2023-10-30T06:35:32.682 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114540.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120894.log 2023-10-30T06:35:32.682 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114540.log.gz 2023-10-30T06:35:32.682 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.126687.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.123790.log 2023-10-30T06:35:32.682 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126687.log.gz 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.120894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112350.log 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120894.log.gz 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124901.log 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123790.log.gz 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91556.log 2023-10-30T06:35:32.683 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112350.log.gz 2023-10-30T06:35:32.684 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98769.log 2023-10-30T06:35:32.684 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124901.log.gz 2023-10-30T06:35:32.684 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130766.log 2023-10-30T06:35:32.684 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91556.log.gz 2023-10-30T06:35:32.684 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.98769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103563.log 2023-10-30T06:35:32.685 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98769.log.gz 2023-10-30T06:35:32.685 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121390.log 2023-10-30T06:35:32.685 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130766.log.gz 2023-10-30T06:35:32.685 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103563.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.128779.log -- replaced with /var/log/ceph/ceph-client.admin.103563.log.gz 2023-10-30T06:35:32.685 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.686 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99895.log 2023-10-30T06:35:32.686 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121390.log.gz 2023-10-30T06:35:32.686 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128779.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.80223.log 2023-10-30T06:35:32.686 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128779.log.gz 2023-10-30T06:35:32.686 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99895.log.gz 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.tmp-client.admin.80223.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.80223.log.gzgzip 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.121296.log 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126280.log 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.121296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91282.log 2023-10-30T06:35:32.687 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121296.log.gz 2023-10-30T06:35:32.688 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118341.log 2023-10-30T06:35:32.688 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.126280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126280.log.gz 2023-10-30T06:35:32.688 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91282.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.104353.log 2023-10-30T06:35:32.688 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91282.log.gz 2023-10-30T06:35:32.688 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121616.log 2023-10-30T06:35:32.689 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118341.log.gz 2023-10-30T06:35:32.689 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123145.log 2023-10-30T06:35:32.689 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104353.log.gz 2023-10-30T06:35:32.689 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124818.log 2023-10-30T06:35:32.689 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121616.log.gz 2023-10-30T06:35:32.690 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.123145.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123145.log.gz 2023-10-30T06:35:32.690 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.admin.99333.log 2023-10-30T06:35:32.690 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124818.log.gz 2023-10-30T06:35:32.690 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92008.log 2023-10-30T06:35:32.690 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99333.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99333.log.gz 2023-10-30T06:35:32.691 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.109527.log 2023-10-30T06:35:32.691 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92008.log.gz 2023-10-30T06:35:32.691 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128839.log 2023-10-30T06:35:32.691 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129706.log 2023-10-30T06:35:32.691 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.109527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109527.log.gz 2023-10-30T06:35:32.692 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119250.log 2023-10-30T06:35:32.692 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128839.log.gz 2023-10-30T06:35:32.692 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132820.log 2023-10-30T06:35:32.692 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129706.log.gz 2023-10-30T06:35:32.692 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90375.log 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119250.log.gz 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132442.log 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132820.log.gz 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90375.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90375.log.gz 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.91158.log 2023-10-30T06:35:32.693 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132442.log.gz 2023-10-30T06:35:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103836.log 2023-10-30T06:35:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91158.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91158.log.gz 2023-10-30T06:35:32.694 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100194.log 2023-10-30T06:35:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103836.log.gz 2023-10-30T06:35:32.694 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119553.log 2023-10-30T06:35:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.100194.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100194.log.gz 2023-10-30T06:35:32.695 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119712.log 2023-10-30T06:35:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119553.log.gz 2023-10-30T06:35:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102582.log 2023-10-30T06:35:32.695 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128517.log 2023-10-30T06:35:32.696 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119712.log.gz 2023-10-30T06:35:32.696 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115034.log 2023-10-30T06:35:32.696 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102582.log.gz 2023-10-30T06:35:32.696 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95818.log 2023-10-30T06:35:32.696 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.128517.log: /var/log/ceph/ceph-client.admin.115034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128517.log.gz 2023-10-30T06:35:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115034.log.gz 2023-10-30T06:35:32.697 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.132316.log 2023-10-30T06:35:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95818.log.gz 2023-10-30T06:35:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132358.log 2023-10-30T06:35:32.697 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121067.log 2023-10-30T06:35:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132316.log: 0.0%/var/log/ceph/ceph-client.admin.132358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112290.log 2023-10-30T06:35:32.698 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.132316.log.gz 2023-10-30T06:35:32.698 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132358.log.gz 2023-10-30T06:35:32.698 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.121067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116252.log 2023-10-30T06:35:32.698 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121067.log.gz 2023-10-30T06:35:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.112290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131458.log 2023-10-30T06:35:32.699 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112290.log.gz 2023-10-30T06:35:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5/var/log/ceph/ceph-client.admin.116252.log: --verbose -- /var/log/ceph/ceph-client.admin.95540.log 2023-10-30T06:35:32.699 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116252.log.gz 2023-10-30T06:35:32.699 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131458.log.gz 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90385.log 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118033.log 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.95540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95540.log.gz 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124817.log 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90385.log.gz 2023-10-30T06:35:32.700 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90680.log 2023-10-30T06:35:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118033.log.gz 2023-10-30T06:35:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124817.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-30T06:35:32.701 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124817.log.gz 2023-10-30T06:35:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129120.log 2023-10-30T06:35:32.701 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90680.log.gz 2023-10-30T06:35:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102702.log 2023-10-30T06:35:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129120.log.gz 2023-10-30T06:35:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104798.log 2023-10-30T06:35:32.702 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102702.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105438.log 2023-10-30T06:35:32.702 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102702.log.gz 2023-10-30T06:35:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92526.log 2023-10-30T06:35:32.703 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104798.log.gz 2023-10-30T06:35:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.105438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123886.log 2023-10-30T06:35:32.703 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105438.log.gz 2023-10-30T06:35:32.703 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93170.log 2023-10-30T06:35:32.704 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92526.log.gz 2023-10-30T06:35:32.704 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130770.log 2023-10-30T06:35:32.704 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123886.log: /var/log/ceph/ceph-client.admin.93170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91422.log 2023-10-30T06:35:32.704 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123886.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93170.log.gz 2023-10-30T06:35:32.704 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130770.log.gz 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90667.log 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91422.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91422.log.gz 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102502.log 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90667.log.gz 2023-10-30T06:35:32.705 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103182.log 2023-10-30T06:35:32.706 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.102502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125085.log 2023-10-30T06:35:32.706 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102502.log.gz 2023-10-30T06:35:32.706 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.103182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116605.log 2023-10-30T06:35:32.706 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103182.log.gz 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91002.log 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125085.log.gz 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116605.log.gzgzip 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.130773.log 2023-10-30T06:35:32.707 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91002.log.gz 2023-10-30T06:35:32.708 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124983.log 2023-10-30T06:35:32.708 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130773.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117385.log 2023-10-30T06:35:32.708 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130773.log.gz 2023-10-30T06:35:32.708 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131980.log 2023-10-30T06:35:32.708 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124983.log: /var/log/ceph/ceph-client.admin.117385.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117385.log.gz 2023-10-30T06:35:32.709 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124983.log.gz 2023-10-30T06:35:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117469.log 2023-10-30T06:35:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131980.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.131980.log.gz -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-30T06:35:32.709 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.709 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86031.log 2023-10-30T06:35:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.117469.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117469.log.gz 2023-10-30T06:35:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.2.log: 93.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-30T06:35:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119836.log 2023-10-30T06:35:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108448.log 2023-10-30T06:35:32.710 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.86031.log: /var/log/ceph/ceph-client.admin.119836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86031.log.gz 2023-10-30T06:35:32.711 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119836.log.gz 2023-10-30T06:35:32.711 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130468.log 2023-10-30T06:35:32.711 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118572.log 2023-10-30T06:35:32.711 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.108448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108448.log.gz 2023-10-30T06:35:32.711 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130468.log.gz 2023-10-30T06:35:32.712 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118116.log 2023-10-30T06:35:32.712 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125954.log 2023-10-30T06:35:32.712 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118116.log: /var/log/ceph/ceph-client.admin.118572.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118572.log.gz 2023-10-30T06:35:32.712 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118116.log.gz 2023-10-30T06:35:32.712 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129749.log 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123988.log 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125954.log.gz 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129749.log.gz 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114865.log 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90162.log 2023-10-30T06:35:32.713 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.114865.log: /var/log/ceph/ceph-client.admin.123988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123988.log.gz 2023-10-30T06:35:32.714 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114865.log.gz 2023-10-30T06:35:32.714 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90012.log 2023-10-30T06:35:32.714 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96398.log 2023-10-30T06:35:32.714 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90162.log.gz 2023-10-30T06:35:32.715 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90012.log.gz 2023-10-30T06:35:32.715 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91482.log 2023-10-30T06:35:32.715 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90830.log 2023-10-30T06:35:32.715 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91482.log: /var/log/ceph/ceph-client.admin.96398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96398.log.gz 2023-10-30T06:35:32.715 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91482.log.gz 2023-10-30T06:35:32.716 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104028.log 2023-10-30T06:35:32.716 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99604.log 2023-10-30T06:35:32.716 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90830.log.gz 2023-10-30T06:35:32.716 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104028.log.gzgzip 2023-10-30T06:35:32.716 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113022.log 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132694.log 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.113022.log: /var/log/ceph/ceph-client.admin.99604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99604.log.gz 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113022.log.gz 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131407.log 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118645.log 2023-10-30T06:35:32.717 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132694.log.gz 2023-10-30T06:35:32.718 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131407.log.gz 2023-10-30T06:35:32.718 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130132.log 2023-10-30T06:35:32.718 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99752.log 2023-10-30T06:35:32.718 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.118645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118645.log.gz 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130132.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92442.log 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130132.log.gz 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92299.log 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.99752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99752.log.gz 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92442.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96860.log 2023-10-30T06:35:32.719 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92442.log.gz 2023-10-30T06:35:32.720 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130216.log 2023-10-30T06:35:32.720 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.92299.log: /var/log/ceph/ceph-client.admin.96860.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92299.log.gz 2023-10-30T06:35:32.720 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96860.log.gz 2023-10-30T06:35:32.720 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104111.log 2023-10-30T06:35:32.720 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126141.log 2023-10-30T06:35:32.721 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.130216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130216.log.gz 2023-10-30T06:35:32.721 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104111.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123461.log 2023-10-30T06:35:32.721 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.721 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116942.log 2023-10-30T06:35:32.721 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.123461.log: /var/log/ceph/ceph-client.admin.126141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126141.log.gz 2023-10-30T06:35:32.722 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123461.log.gz 2023-10-30T06:35:32.722 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125185.log 2023-10-30T06:35:32.722 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99132.log 2023-10-30T06:35:32.722 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116942.log.gz 2023-10-30T06:35:32.722 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125185.log.gz 2023-10-30T06:35:32.723 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104659.log 2023-10-30T06:35:32.723 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91262.log 2023-10-30T06:35:32.723 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.104659.log: /var/log/ceph/ceph-client.admin.99132.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99132.log.gz 2023-10-30T06:35:32.723 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104659.log.gz 2023-10-30T06:35:32.723 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131044.log 2023-10-30T06:35:32.724 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119109.log 2023-10-30T06:35:32.724 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91262.log.gz 2023-10-30T06:35:32.724 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.131044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131044.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91116.log 2023-10-30T06:35:32.724 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.724 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127353.log 2023-10-30T06:35:32.725 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.91116.log: /var/log/ceph/ceph-client.admin.119109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119109.log.gz 2023-10-30T06:35:32.725 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91116.log.gz 2023-10-30T06:35:32.725 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132484.log 2023-10-30T06:35:32.725 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125490.log 2023-10-30T06:35:32.725 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.127353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127353.log.gz 2023-10-30T06:35:32.726 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.132484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132484.log.gz 2023-10-30T06:35:32.726 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129526.log 2023-10-30T06:35:32.726 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116879.log 2023-10-30T06:35:32.726 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.125490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125490.log.gz 2023-10-30T06:35:32.726 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129526.log.gz 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129100.log 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116060.log 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.129100.log: /var/log/ceph/ceph-client.admin.116879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116879.log.gz 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129100.log.gz 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90383.log 2023-10-30T06:35:32.727 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130775.log 2023-10-30T06:35:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.116060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116060.log.gz 2023-10-30T06:35:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.90383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90383.log.gz 2023-10-30T06:35:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-30T06:35:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124900.log 2023-10-30T06:35:32.728 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.130775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130775.log.gz 2023-10-30T06:35:32.729 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119027.log 2023-10-30T06:35:32.729 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.124900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124900.log.gz 2023-10-30T06:35:32.729 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.119027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119027.log.gz 2023-10-30T06:35:32.731 INFO:teuthology.orchestra.run.smithi079.stderr: 90.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-30T06:35:32.731 INFO:teuthology.orchestra.run.smithi079.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-30T06:35:32.731 INFO:teuthology.orchestra.run.smithi079.stderr: 90.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-30T06:35:32.731 INFO:teuthology.orchestra.run.smithi079.stderr: 90.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-30T06:35:32.731 INFO:teuthology.orchestra.run.smithi079.stderr: 90.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-30T06:35:32.732 INFO:teuthology.orchestra.run.smithi112.stderr: 90.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-30T06:35:32.732 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-30T06:35:32.733 INFO:teuthology.orchestra.run.smithi112.stderr: 90.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-30T06:35:32.733 INFO:teuthology.orchestra.run.smithi112.stderr: 90.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-30T06:35:32.733 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-30T06:35:32.733 INFO:teuthology.orchestra.run.smithi112.stderr: 2023-10-30T06:35:32.733 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m0.287s 2023-10-30T06:35:32.734 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m0.876s 2023-10-30T06:35:32.734 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.044s 2023-10-30T06:35:32.753 INFO:teuthology.orchestra.run.smithi079.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-30T06:35:32.800 INFO:teuthology.orchestra.run.smithi079.stderr: 89.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-30T06:35:32.802 INFO:teuthology.orchestra.run.smithi079.stderr: 2023-10-30T06:35:32.802 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m0.436s 2023-10-30T06:35:32.802 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m1.669s 2023-10-30T06:35:32.802 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.217s 2023-10-30T06:35:32.803 INFO:tasks.ceph:Archiving logs... 2023-10-30T06:35:32.803 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi079/log 2023-10-30T06:35:32.804 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T06:35:33.457 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi112/log 2023-10-30T06:35:33.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-30T06:35:33.625 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 1957, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 368, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-10-30T06:35:33.625 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-30T06:35:33.645 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 1957, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d0db9f2f95b9d6344950185071cace2a50d296eb/qa/tasks/ceph.py", line 368, in valgrind_post raise valgrind_exception Exception: saw valgrind issues 2023-10-30T06:35:33.646 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-30T06:35:33.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T06:35:33.649 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-30T06:35:33.729 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-30T06:35:33.738 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-30T06:35:33.738 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T06:35:33.741 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-30T06:35:33.754 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2023-10-30T06:35:33.755 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2023-10-30T06:35:33.758 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-30T06:35:33.758 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2023-10-30T06:35:33.758 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 +1352us[+1352us] +/- 72ms 2023-10-30T06:35:33.759 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv02.front.sepia.ceph.com 3 7 377 63 -842us[ -866us] +/- 61ms 2023-10-30T06:35:33.759 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -86us[ -86us] +/- 91ms 2023-10-30T06:35:33.759 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:35:33.760 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-30T06:35:33.761 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-10-30T06:35:33.761 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 1 +1364us[+1364us] +/- 72ms 2023-10-30T06:35:33.761 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -872us[ -869us] +/- 62ms 2023-10-30T06:35:33.761 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -72us[ -72us] +/- 91ms 2023-10-30T06:35:33.762 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-30T06:35:33.762 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-30T06:35:33.773 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-30T06:35:33.774 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-30T06:35:33.782 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-30T06:35:33.858 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-30T06:35:33.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-30T06:35:33.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:35:33.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-30T06:35:33.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:35:33.969 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-30T06:35:33.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-30T06:35:33.988 INFO:teuthology.task.internal:Duration was 1755.453836 seconds 2023-10-30T06:35:33.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-30T06:35:33.997 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-30T06:35:33.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T06:35:34.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-30T06:35:34.041 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-30T06:35:34.052 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-30T06:35:34.206 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-30T06:35:34.207 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2023-10-30T06:35:34.208 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T06:35:34.266 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2023-10-30T06:35:34.266 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-30T06:35:34.299 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-30T06:35:34.299 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T06:35:34.307 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-30T06:35:34.367 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-30T06:35:34.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T06:35:34.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-30T06:35:34.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-30T06:35:34.548 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-30T06:35:34.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T06:35:34.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-30T06:35:34.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-30T06:35:34.614 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T06:35:34.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-30T06:35:34.641 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2023-10-30T06:35:34.668 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2023-10-30T06:35:34.691 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T06:35:34.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:35:34.721 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-30T06:35:34.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-30T06:35:34.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-30T06:35:34.761 INFO:teuthology.task.internal:Transferring archived files... 2023-10-30T06:35:34.762 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi079 2023-10-30T06:35:34.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T06:35:34.796 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005/remote/smithi112 2023-10-30T06:35:34.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-30T06:35:34.829 INFO:teuthology.task.internal:Removing archive directory... 2023-10-30T06:35:34.830 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T06:35:34.833 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-30T06:35:34.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-30T06:35:34.890 INFO:teuthology.task.internal:Not uploading archives. 2023-10-30T06:35:34.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-30T06:35:34.901 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-30T06:35:34.901 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T06:35:34.904 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-30T06:35:34.915 INFO:teuthology.orchestra.run.smithi079.stdout: 265243 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 06:35 /home/ubuntu/cephtest 2023-10-30T06:35:34.933 INFO:teuthology.orchestra.run.smithi112.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 30 06:35 /home/ubuntu/cephtest 2023-10-30T06:35:34.934 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-30T06:35:34.947 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-30T06:35:34.959 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: saw valgrind issues 2023-10-30T06:35:34.959 INFO:teuthology.nuke:Checking targets against current locks 2023-10-30T06:35:34.985 DEBUG:teuthology.nuke:shortname: smithi079 2023-10-30T06:35:34.985 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T06:35:35.008 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 05:57:54.528055', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOy9njApuL9486j3mpSSWwFrt0UsaVb0q/6Gn4YBG+PqZmwquojCA12rs3VU4DVsLcQDMY61NwRuS+U4/5jWhEU='} 2023-10-30T06:35:35.015 DEBUG:teuthology.nuke:shortname: smithi112 2023-10-30T06:35:35.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-30T06:35:35.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2023-10-30_05:47:25-rados-main-distro-default-smithi/7441005', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-30 05:57:54.529735', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf6npLNQ3Gfa1m3akue2LinVUVmcPcQi3rhjOXz5RrAMVi0QhDKvV/Xf1DeWYnn3MpsGFcqGrbCo/2Pe0QXkCw='} 2023-10-30T06:35:35.058 INFO:teuthology.orchestra.console.smithi079:Power off 2023-10-30T06:35:35.059 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T06:35:35.084 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-10-30T06:35:35.085 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:35.103 INFO:teuthology.orchestra.console.smithi112:Power off 2023-10-30T06:35:35.104 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-30T06:35:35.117 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-30T06:35:35.130 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2023-10-30T06:35:35.130 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:35.155 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2023-10-30T06:35:39.119 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:39.156 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:39.229 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-30T06:35:39.266 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2023-10-30T06:35:43.231 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:43.267 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:43.342 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-30T06:35:43.377 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-10-30T06:35:43.478 INFO:teuthology.orchestra.console.smithi112:Power off completed 2023-10-30T06:35:47.347 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-30T06:35:47.457 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-10-30T06:35:47.558 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-10-30T06:35:47.662 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/valgrind} duration: 1755.4538359642029 failure_reason: saw valgrind issues flavor: default owner: scheduled_sjust@teuthology status: fail success: false 2023-10-30T06:35:47.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-30T06:35:47.749 INFO:teuthology.run:FAIL